CAP $6629$: Reinforcement Learning Spring $2024$

Course project $2$

Submission: Two files $($one report in $.$pdf and one $.$ipynb$/$code$).$

Please follow the project report guidelines and submit the report with setup, results and

analysis.

In project $1,$ you may realize that when you have a large grid world maze setup, it takes a long

time for the agent to learn a value table. One way to eliminate this challenge is to use neural

networks to approximate the value function. There are two options provided below and you may

choose either one to implement.

Based on your results in project $1,$ you can choose to build a neural network $($or deep

neural network$)$ to approximate your obtained Q or V table.

You can design another complex grid world example and develop the QNN $($or deep

QNN$)$ method based on that.

Either way, you are using a neural network to generate your Q or V value so that you can guide

the agent to move to achieve the goal.

Report requirments:

Maze Description: Design your own grid world example and describe it at the beginning

of the report.

Problem Formulation: Define your states, actions, and rewards.

Q Network Design: Design and implement your Q network.

Pseudo Code: Provide the pseudo code in the report.

Results and Discussions: Show the convergence process of mean square error $($objective

function$)$ and the weights trajectories.

Reference: cite all your reference here.