Problem $2.$

For the system defined in problem $1,$ perform matrix$-$form Value Iteration

method with $V_0(s)=0,=0\mathrm{.}9$ and $=0\mathrm{.}5$ to compute $V^{**}$ and $P^{**}.$Problem $1.$

consider the following system with the state space $S=\{A,B\},$ and action

space $A=\{a^1,a^2\}.$ The state transition diagram is shown below, where

$P(s^{\text{'}}=B|S=A,a=a^{\text{'}})=0\mathrm{.}8,P(S^{\text{'}}=A|s=A,a=a^{\text{'}})=0\mathrm{.}2.$

The reward is as follows: $+2$ moving $to$ state $B$

$0$ maing $to$ state $A$

$-1\mathrm{.}5$ taking action $a^{\text{'}}$

$-1$ taking action $a^2$

a$)$ Construct transition matrices $M(a^{\text{'}}),(a^2)$ and compute $R_s^{a^{\text{'}}},R_s^{a^2}.$

b$)$ Perform matrix$-$form palicy Iteration method with initial 'Palicy

${}^{\text{'}}(A)=a^2,x_{(B)}^{\text{'}}=a^{\text{'}}$ and $=0\mathrm{.}9$ to compute $x^{**}.$