For the $44$ grid world example we discussed in the lecture:

Consider $=1($undiscounted MDP$)$

Non$-$terminal states: $1,2,3,$dots,$14$

Two terminal states $($shaded squares$)$

Actions leading out of the grid leave the state unchanged.

The reward is $-1$ for all transitions until the terminal state is reached.

The agent follows a policy given as below $($NOT the same as we discussed in the lecture$)$:

$($ North $|s)=30\%,($ South $|s)=20\%,($ West $|s)=40\%,($ East $|s)=10\%$