Please solveQ$1$ Solving MDPs

Consider the gridworld MDP for which Left and Right actions are $100\%$ successful. Specifically, the available

actions in each state are to move to the neighboring grid squares. From state $a,$ there is also an exit action

available, which results in going to the terminal state and collecting a reward of $10.$ Similarly, in state $e,$ the

reward for the exit action is $1.$ Exit actions are successful $100\%$ of the time.

Let the discount factor $=1.$ Fil$1$ in the following quantities.

$V_0(d)=|$

$V_1(d)=|$

$V_2(d)=$

$V_3(d)=$

$V_4(d)=|$

$V_5(d)=$