Value: For a Markov decision process with $6$ regular states $($s$1-$s$6)$ and $2$ terminal states at either end, the action at each non$-$terminal state is aR$($s$,$left$)=$R$($s$,$right$)=$R$($s$,\backslash$deg $).$ Suppose $\backslash$gamma $=1$ and

Discount factor: $\backslash$gamma $=1$

Value fn:

$\backslash$table$[[,],[,],[,],[,]]$

Policy: a random move to its neighborhoods.

Calculate the transition matrix.

Calculate all the state value functions under this policy.