ASSIGNMENT No. 3

Complex Computing Problem

Objective:

Reinforcement learning, in which an agent learns to make decisions by interacting with the environment, has really taken off in the last few years. It is one of the hottest topics in artificial intelligence and machine learning these days, and research in this domain is progressing at a fast pace. In reinforcement learning (RL), an agent converts their actions and experiences into learning to make better decisions in the future. The purpose of this assignment is to apply reinforcement learning to autonomous self-driving cars.

Scenario: Autonomous Self-Driving Car Through Reinforcement Learning

This problem statement outlines the task of implementing an autonomous self-driving racing car using deep Q networks. Let's break down the key components and concepts involved:

Environment and Framework:

o The racing track and its surroundings serve as the environment.

o OpenAI Gym CarRacing-v0 framework is used as the simulation environment.

Agent:

o The racing car and its driver are considered the agent in this scenario.

o The agent's goal is to learn how to drive autonomously on the racing track.

Learning Approach:

o Deep Q networks (DQN) will be employed for the learning process.

o DQN is a type of reinforcement learning algorithm that uses a neural network to approximate the Q-values, which represent the expected future rewards for taking a particular action in a given state.

Observations (States):

o The states are represented by images captured from a camera mounted in front of the car.

o These images provide the necessary information for the agent to make decisions about its actions.

Actions:

o The agent interacts with the environment by taking actions.

o Actions are represented by a three-dimensional vector aR3, where:

The first component is for turning left.

The second component is for moving forward.

The third component is for turning right.

Rewards:

o The environment provides rewards to the agent based on its actions.

o Rewards serve as feedback for the agent's behavior, helping it learn to navigate the racing track effectively.

Training Data:

o The agent's interactions with the environment are recorded as tuples.

o These tuples, typically in the form (state, action,reward, next state), act as training data for the deep Q network.

o The training process involves updating the Q-values in the neural network based on the received rewards and the predicted future rewards.

Output:

o The ultimate output is for the autonomous racing car to learn a policy that enables it to navigate the racing track successfully by making appropriate decisions based on the visual input from the camera.

In summary, the problem involves using reinforcement learning, specifically deep Q networks, to train an autonomous racing car to drive on a simulated racing track. The learning process relies on the agent's interactions with the environment, where states (captured images), actions, and rewards form the basis of training the neural network. The goal is to develop a policy that allows the agent to drive autonomously and make optimal decisions while navigating the track.

Deliverables

a) Develop a Python or Java or C++/ code for the reinforcement learning algorithms for the self-driving car.

b) Explain and Examine the results and infer the gradual progress of the reinforcement learning algorithms used for automated self-driving cars.

1.	Discussion of related techniques and algorithms.
2.	Implementation of algorithms [Java/C++/Python], analysis, and evaluation of results of proposed algorithms