$2.$ Obtaining optimal Cost$-$to$-$Go values using Bellman's equations

$20$ Marks

Please obtain the Optimal Cost$-$to$-$Go values using the Bellman's equations for Problem $1.$ Please note that this will involve successively solving coupled systems of

equations as was discussed in the class. Please match your results with that obtained for Problem $1$ using the computational method. $1.$ Obtaining optimal Cost$-$to$-$Go values using value$/$policy iteration method $($to be done in group$)$

$20$ Marks

Please write down a computer program in Matlab to obtain the Optimal Cost$-$to$-$Go values $($using value or policy iteration method$)$ for the stochastic robot navigation problem discussed in the class. The following figure shows the states. Cell showing $"+1"$ is the goal, cell showing $"-1"$ needs to be avoided. These values of the cells have been pre$-$determined. The shaded cell represents a wall in the cell. Also, consider that an action by the robot in a direction results in motion in that direction with probability $0\mathrm{.}8$ and motion in directions perpendicular to that intended direction with $0\mathrm{.}1$ each. If the robot hits a wall, it comes back to original position. Also, every step taken by the robot has a cost associated of $\backslash (1/25\backslash )$ units.