For the MDP above $($same as the one we had in class$),$ we randomly selected a policy and generated four $(4)$ episode.

What will be the values after each episode if we use the Model$-$free Monte Carlo method. You should write down the two $(2)$ utility values for each question. $($VAns$($sans$)=?$ and VQuit$($squit$)=?).$

i$)$ Policy $=$ Ans, Data $=$ sstart; Ans, $4,$ sstart; Ans, $4,$ send

ii$)$ Policy $=$ Quit, Data $=$ sstart; Quit, $10,$ send

iii$)$ Policy $=$ Ans, Data $=$ sstart; Ans, $4,$ send

iv$)$ Policy $=$ Ans, Data $=$ sstart; Ans, $4,$ sstart; Ans, $4,$ sstart; Ans, $4,$ send v$)$ Policy $=$ Quit, Data $=$ sstart; Quit, $10,$ sendQuestion $1$

$\{$:$s_{\text{a}\text{n}\text{s}})=$

For the MDP above $($same as the one we had in class$),$ we randomly selected a

policy and generated four $(4)$ episode.

What will be the values after each episode if we use the Model$-$free Monte Carlo

method. You should write down the two $(2)$ utility values for each question.

$?$ and $V_{\text{Q}\text{u}\text{i}\text{t}}(s_{\text{q}\text{u}\text{i}\text{t}})=?).$

i$)$ Policy $=$Ans, Data $=s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{e}\text{n}\text{d}}$

ii$)$ Policy $=$ Quit, Data $=s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$; Quit, $10,s_{\text{e}\text{n}\text{d}}$

iii$)$ Policy $=$Ans, Data $=s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{e}\text{n}\text{d}}$

iv$)$ Policy $=$Ans, Data $=s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$;Ans,$4,s_{\text{e}\text{n}\text{d}}$

v$)$ Policy $=$ Quit, Data $=s_{\text{s}\text{t}\text{a}\text{r}\text{t}}$; Quit $,10,s_{\text{e}\text{n}\text{d}}$