Question 1. [8 Marks] Consider a Markov chain with 5 states {0, 1, 2, 3, 4}...

Transcribed Image Text:

Question 1. [8 Marks] Consider a Markov chain with 5 states {0, 1, 2, 3, 4} and the one- step transition matrix P given by P= = 1/4 1/3 0 0 1 1/2 0 1/3 0 0 1 0 0 1/2 0 0 1/4 0 0 0 0 0 1/3 0 1/2 (a) Draw the transition diagram for this Markov chain and identify the communicating classes. Which states are transient and which states are recurrent? (b) For each transient state in the state space, evaluate the expected time until the Markov chain first hits a recurrent state. (c) Find the general form of the stationary distribution for this chain. Question 1. [8 Marks] Consider a Markov chain with 5 states {0, 1, 2, 3, 4} and the one- step transition matrix P given by P= = 1/4 1/3 0 0 1 1/2 0 1/3 0 0 1 0 0 1/2 0 0 1/4 0 0 0 0 0 1/3 0 1/2 (a) Draw the transition diagram for this Markov chain and identify the communicating classes. Which states are transient and which states are recurrent? (b) For each transient state in the state space, evaluate the expected time until the Markov chain first hits a recurrent state. (c) Find the general form of the stationary distribution for this chain.

Answer rating: 100% (QA)

Question a The state transition probability matrix is given using which the state transition diagram ... View the full answer