Consider tic$-$tac$-$toe to answer this

question. Assume that states are numbered from S$1$ to Sn$.$

$($a$)$ List the four elements of reinforcement learning and write one well$-$articulated formal

statement explaining the role of each element. $[2$ M$]$

$($b$)$ Write the temporal difference rule for learning each state's value. $[0\mathrm{.}5$ M$].$ Explain various

elements and the workings of this rule. $[1\mathrm{.}5$ M$]$

$($c$)$ Let the value of the current state be $4\mathrm{.}5,$ and all its possible successor$/$predecessor states

have a value of $2\mathrm{.}7.$ Use $0\mathrm{.}9$ to be the parameter value for any parameter you need to use

to solve this problem. Given this, Revise the estimate of the value of the current state using

your answer to $($b$).$ Explain your answer $[2$ M$]$

$[$