You are given an environment with one state, X$,$ and two actions, b and c$.$ T is the terminal

state. Your Temporal Difference $($TD$)$ algorithm generates the following episode using the policy when

interacting with its environment:

Timestep Reward State Action

$0$ X b

$116$ X c

$212$ X b

$316$ T

$$ The policy is given by: $($b$|$X$)=0\mathrm{.}9,($c$|$X$)=0\mathrm{.}1.$

$$ The current values of q are: q$($X$,$ b$)=1$ and q$($X$,$ c$)=2.$

$$ The discount factor,

$=1$

$2.$

$$ The step size, $=0\mathrm{.}1.$

Show the values of q$($X$,$ b$)$ and q$($X$,$ c$)$ after their first update using the following approaches:

$($a$)1-$step SARSA $(2+2)$

$($b$)2-$step SARSA $(2+2)$

$($c$)2-$step Full Tree Backup $(3+3)$

Note: You should update q$($X$,$ b$)$ and q$($X$,$ c$)$ only once per learning algorithm. Show your work and carry

out your calculations to two decimal places.