We follow the steps of the Policy Iteration algorithm as explained in the class.

$1.$ Write down the Bellman equation.

$2.$ The initial policy is $\backslash$pi $($A$)=1$ and $\backslash$pi $($B$)=1.$ That means that action $1$ is taken when in state A$,$ and the same action

is taken when in state B as well. Calculate the values V

$\backslash$pi

$2$

$($A$)$ and V

$\backslash$pi

$2$

$($B$)$ from two iterations of policy evaluation

$($Bellman equation$)$ after initializing both V

$\backslash$pi

$0$

$($A$)$ and V

$\backslash$pi

$0$

$($B$)$ to $0.$