Consider a Q$-$learning agent, in a world containing actions $\{$T rain, Rest$\}$ and states $\{$W eekday, W eekend$\}.$ Suppose the Q$-$values are currently, at time t$,$ as follows.

Q$[$W eekday, T rain$]=8,$ Q$[$W eekday, Rest$]=10$

Q$[$W eekend, T rain$]=15,$ Q$[$W eekend, Rest$]=4$

Assume learning rate $\backslash$alpha $=0\mathrm{.}1$ and discount $\backslash$gamma $=0\mathrm{.}95.$ Suppose that at time t the agent is in state W eekday.

i$.$ If the agent uses $\backslash$epsi $-$greedy exploration with $\backslash$epsi $=0\mathrm{.}01,$ what is the probability of choosing action T rain? $[1]$

ii$.$ Suppose instead that the agent uses softmax action selection with $\backslash$tau $=0\mathrm{.}9.$ What is the probability of choosing action T rain? $[2]$

iii. IfattimettheagentperformsactionTrain,receivesreward$15$andendsinstate W eekday, how is the table of Q$-$values updated using Q$-$learning? $[2]$