$1.$When the learning rate is $0\mathrm{.}1,$ and the discount factor is $0\mathrm{.}9.$ Consider the following trajectories.

$($s$2,$ north, s$3,$r$=-0\mathrm{.}1)$;

$($s$3,$ east, s$3,$ r$=-0\mathrm{.}1)$;

$($s$3,$ east, s$4,$ r $=-0\mathrm{.}1)$;

$($s$4,$ north, s$4,$ r $=1\mathrm{.}0)$;

Use the Q$-$learning algorithm to help the agent update its $Q$ function: $Q(s,a).$ Please list the $Q$ values that have been changed after each of the four actions. Further, the updated $Q$ value$($s$)$ should be used in computing the $Q$ value update of follow$-$up actions.