Suppose we are learning $Q^{**}(s,a)$ for Pacman's world.

Pacman can take the following actions

$\{N,S,E,W\}$

Currently, Pacman's estimate is $Q(s,a)$ such that for all $s$

$Q(s,N)=10,Q(s,S)=-10,Q(s,E)=5,Q(s,W)=2$

Suppose Pacmans scheme for exploration is to

take a random action with probability $lon=0\mathrm{.}2$

act according to the current policy $(s)=argma{x}_{a}Q(s,a),$ with probability $1-lon=0\mathrm{.}8$

What is the probability of Pacman moving north, i$.$e$.,$ taking action $N?$

Suppose Pacman updates the $Q(s,a)$ estimate using a running average with parameter $=0\mathrm{.}1.$

If Pacman moves south, i$.$e$.,$ makes the action $S$ and receives a reward of $100$ what is the new estimate of $Q(s,a)?$

$Q(s,N)=$

$Q(s,S)=$

$Q(s,E)=$

$Q(s,W)=$