a) What is a Markov Decision Process? Explain in your own words (200 words or less)....

 

Transcribed Image Text:

a) What is a Markov Decision Process? Explain in your own words (200 words or less). b) Consider the state diagram provided in the figure below. Clearly state the states, actions, transition probabilities, rewards, and terminal states present in the diagram. +1 Slow a +1 1.0 Cool +1 0.5 Slow 0.5 Fast 0.5 +2 Warm 0.5 +2 Fast 1.0 -10 Overheated c) Use Value Iteration to find the value for each of the states after k=3 timesteps, assuming a discount factor of i) y = 1.0, and ii) y = 0.xy, where xy are the last two digits of your 7-digit Student ID (e.g., if your ID is '1234567', you must use a discount factor of 0.67) d) Briefly explain why Policy Iteration may converge sooner than Value Iteration for the same MDP (200 words or less). a) What is a Markov Decision Process? Explain in your own words (200 words or less). b) Consider the state diagram provided in the figure below. Clearly state the states, actions, transition probabilities, rewards, and terminal states present in the diagram. +1 Slow a +1 1.0 Cool +1 0.5 Slow 0.5 Fast 0.5 +2 Warm 0.5 +2 Fast 1.0 -10 Overheated c) Use Value Iteration to find the value for each of the states after k=3 timesteps, assuming a discount factor of i) y = 1.0, and ii) y = 0.xy, where xy are the last two digits of your 7-digit Student ID (e.g., if your ID is '1234567', you must use a discount factor of 0.67) d) Briefly explain why Policy Iteration may converge sooner than Value Iteration for the same MDP (200 words or less).