Consider the grid$-$world given below and Pacman who is trying to learn the optimal policy. If an action results in landing into one of the shaded states the corresponding reward is awarded during that transition. All shaded states are terminal states, i$.$e$.,$ the MDP terminates once arrived in a shaded state. The other states have the North, East.

South, West actions available, which deterministically move Pacman to the corresponding neighboring state $($or have

Pacman stay in place if the action tries to move out of the grad$).$ Assume the discount factor $7=0\mathrm{.}5$ and the

Q$-$learning rate a $=0\mathrm{.}5$ for all calculations. Pacman starts in state $(1,3).$