2. Specify the classes, and determine whether they are transient or recurrent for a Markov chain with the transition probability matrix a) P where b) P2 c) P3 d) P4 0 0 0 1 P2 0 0 0 1 12-20 120 1212 P = 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 00 0 0 0 0 34 12 1412 0 0 0 0 0 1212 1212 P4= 0 0 1 0 0 121412 0 12 0 12 0 0 121412 O P3 0 0 0 Problem 2 (a) Classes: {0, 1, 2}. Since irreducible and finite chain, all states are recurrent (b) Classes: {0, 1, 2, 3}. Since irreducible and finite chain, all states are recurrent (c) Classes: {0,2}, {1}, {3,4}. fo = 1 P(To = |X0 = 0) = 1 0.5 lim 0.5" = 1 then 2 is also recurrent f = P(T < |X0 = 1) = 0.5 < 1 1 is transient 0 is recurrent, and = P(T3 = |X = 3) = 1 0.5 limn 0.5" = 1 => 3 is recurrent, and then 4. is also recurrent (d) Classes: {0, 1}, {2}, {3}, {4}. fo = 1 P(T = |X = 0) = 1 limn() = 1 >> 1 is also recurrent f2 = P(T2 < | X0 = 2) = 1 >> 2 is recurrent = P(T3 < |X0 = 3) = }} < 1 >> 3 is transient 0 is recurrent, and then f = 1-P(T = 0|X0 = 4) = 1-1=0