A gridworld is shown in Figure $($a$)$ below. The states are

grid squares, identified by their row and column number $($row first$).$ The

agent always starts in state $(1,1),$ marked with the letter S$.$ There are two

terminal goal states, $(2,3)$ with reward $+5$ and $(1,3)$ with reward $-5.$ Rewards

are $0$ in non$-$terminal states. The reward for a state is received as the agent

moves into the state. The transition function is such that the intended agent

movement $($North$,$ South, West, or East$)$ happens with probability $0\mathrm{.}8.$ With

probability $0\mathrm{.}1$ each, the agent ends up in one of the states perpendicular

to the intended direction. If a collision with a wall happens, the agent stays

in the same state.