Question $5.(4$ marks$)$

Consider a Markov Decision Process $($MDP$)$ with set of states $S$ and set of actions $A.$

Explain what is meant by:

$($a$)(1$ mark$)$ A policy $.$

$($b$)(1$ mark$)$ The discounted return for an episode.

$($c$)(1$ mark$)$ The value of a state $s$ with respect to a policy $,$ i$.$e$.,V^{}(s).$

$($d$)(1$ mark$)V^{{}^{**}}(s),$ for a state $s.$