$($a$)$ Write down a scenario where you will find Monte Carlo methods more appropriate than dynamic

programming$-$based solutions.

$[0\mathrm{.}5$ Mark$]$

Also, write down a scenario when dynamic programming solutions are preferred over Monte Carlo$-$based

solutions. Mark$]$

$(6)$ Write down any two ways $($with reasoring$)$ that increase the coverage of states by episodes being

used to learn solutions based on Mornte Carlo, $[2$ Marks$][$Precise statements$]$

$($ctic$)$ when $4=02.[1\mathrm{.}5$ Marks$]($General expressions are discouraged. Provide an answer that

exactiy matches this scenario$)$

Ferocm polticy to tiansion betwien the two states The goal is to estimate the value function V$($S$1)$

Wre First Visit Monte Carle

Shates: