Another problem type excellent for reinforcement learning is the so-called gridworld. We present a simple 4 × 4 gridworld in Figure 10.26. The two greyed corners are the desired terminal states for the agent. From all other states, agent movement is either up, down, left, or right. The agent cannot move off the grid: attempting to leaves the state unchanged. The reward for all transitions, except to the terminal states, is −1. Work through a sequence of grids that produce a solution based on the temporal difference algorithm presented in Section 10.7.2.