$3.[20$ points$]$ Programming Assignment I:

For this assignment, you are going to implement a simple simulation of robot path plan$-$ ning on a grid world and use value iteration to come up with a policy to get the robot to navigate a maze. You are given two files $($mdp$.$py and utils$2.$py$)$ with an MDP implementa$-$ tion that you need to understand.

$($a$)$ Run the code as follows:

python mdp$.$py

Show a snapshot of the code running.

$($b$)$ Explain the relationship between the class MDP and the class GridMDP in mdp$.$py$.$

$($c$)$ Explain the following python functions from the sample code:

$-$ vector add

$-$ turn right

$-$ turn left

$-$ value iteration

$-$ best policy

$-$ expected utility

$($d$)$ What happens with the found policy if the reward for the entries that have $-0\mathrm{.}04$ are changed to $\backslash (-1,-0\mathrm{.}3\backslash ),$ or $-0\mathrm{.}02?$ Explain the impact of changing rewards on the policy.