Gridworld $-$ Q Learning

Create a $55$ grid world

An agent to move around

Four possible actions

Have a goal state.

Reward a Goal $=5$ and Another

terminal state $=-5$

Elsewhere Reward $=0$

Any action that takes you outside

boundary, Reward $=-1$

Run $100,000$ episodes

Keep a random no$.$ seed

Plot the converged policy and value function for this grid world.

Do it for $=0\mathrm{.}1,0\mathrm{.}5$ and $0\mathrm{.}9,$ take epsilon $=0\mathrm{.}1.$

For gamma $=0\mathrm{.}9,$ plot the no$.$ of steps to reach the goal across

episodes for epsilon $=0\mathrm{.}1,0\mathrm{.}3$ and $0\mathrm{.}5.$

For all the above, keep the learning rate alpha $=0\mathrm{.}1.$