Consider a grid$-$world problem as shown in Figure $1.$ The four possible actions are north, south, east, and west and they are deterministic including for points A and B$.$ If the action would take the agent off the grid: no move but reward $=-1$ will be obtained. Other actions produce reward $=0,$ except actions that move the agent out of special states A and B as shown.

Actions

Figure $1$: Grid World Problem

Formulate a Markov decision process problem for this grid world.

Find the optimal value functions using the iterative policy evaluation algorithm.

Find the optimal policy using policy iteration and policy improvement algorithm.

Find the optimal policy using the value iteration algorithm.

Compare your results with the results obtained in the book.

please solve it proper explanation.