Problem $2$: Policy Evaluation $(25$ points$)$

In problem $2$ you will implement policy evaluation as follows

$V^{}(s)={\displaystyle \underset{s^{\text{'}}}{\overset{?}{}}}T(s,(s),s^{\text{'}})[R(s,(s),s^{\text{'}})+V^{}(s^{\text{'}})]$

This time we have discounting and we also introduce a new variable for the number of iterations. Here is the first test case.Note that there is no randomness involved this time and that we use discounting. As usual, your first task is to implement the parsing of this grid MDP in the function read$\_$grid$\_$mdp$\_$problem$\_$p$2($file$\_$path$)$ of the file parse.py$.$ You may use any appropriate data structure.

Next you implement value iteration for policy evaluation as discussed in class. Your policy$\_$evaluation$($problem$)$ function in

p$2.$py should return the evolution of values as follows.

This example should look familiar. We have covered it in chapter $2$ of our lecture slides.

Hint: The output of an individual floating point value $v$ was done as follows

return$\_$value $+=|\{$:$7\mathrm{.}2f\}|?.$ format$($v$)$

Finally, check the correctness of your implementation via