Markov chain transition

2. A Markov chain with state space {1, 2, 3} has transition probability matrix 00 0.3 0.1 a: 0.3 0.3 0.4 0.4 0.1 0.5 (a) Is this Markov chain irreducible? Is the Markov chain recurrent or transient? Explain your answers. (b) What is the period of state 1? Hence deduce the period of the remaining states. Does this Markov chain have a limiting distribution? (c) Consider a general three-state Markov chain with transition matrix 3011 3012 1013 P = P21 P22 P23 1031 P32 P33 Give an example of a specic set of probabilities jag-'3; for which the Markov chain is not irreducible (there is no single right answer to this1 of course l]. Problem 3. Consider the Markov chain shown in Figure 2. Figure 2: Problem 3 Markov chain 1. Let the initial distribution be Pr(A) : Pr(B) = 0.5. What is the probability distribution after one step? 2. What is the stationary distribution of the Markov chain? 1. Consider the Markov chain with the following transition matrix. 0 0.5 0.5 0.5 0 0.5 0.5 0.5 0 (a) Draw the transition diagram of the Markov chain. (b) Is the Markov chain ergodic? Give a reason for your answer. (c) Compute the two step transition matrix of the Markov chain. (d) What is the state distribution *2 for t = 2 if the initial state distribution for t = 0 is no = (0.1, 0.5, 0.4) ?Problem 7.4 (10 points) A Markov chain X0,X1, X2, . . . with state space S = {1, 2, 3, 4} has the following transition graph: (a) Provide the transitiOn matrix fer the Markov chain. (13) Determine all recurrent and all transient states