b

$3$ points possible $($graded$)$

If we initialize the value function with $0,$ enter the value of state $B$ after:

one value iteration, $V_{B1}^{*}$

two value iterations, $V_{B2}^{*}$

infinite value iterations, $V_B^{*}$

You have used $0$ of $3$ attempts

C

$1$ point possible $($graded$)$

Select all that are true

In an MDP$,$ the optimal policy for a given state $s$ is unique

The problem of determining the value of a state is solved recursively by value iteration algorithm

For a given MDP$,$ the value function $V^{*}(s)$ of each state is known a priori

$V^{*}(s)={\displaystyle \underset{s^{\text{'}}}{\overset{?}{}}}T(s,a,s^{\text{'}})[R(s,a,s^{\text{'}})+V^{*}(s^{\text{'}})]$

$Q^{*}(s,a)={\displaystyle \underset{s^{\text{'}}}{\overset{?}{}}}T(s,a,s^{\text{'}})[R(s,a,s^{\text{'}})+V^{*}(s^{\text{'}})]$