Transition Probability

A Markov chain with state space {1, 2, 3} has transition probability matrix 0.6 0.3 0.1\\ P. = 0.3 0.3 0.4 0.4 0.1 0.5 (a) Is this Markov chain irreducible? Is the Markov chain recurrent or transient? Explain your answers. (b) What is the period of state 1? Hence deduce the period of the remaining states. Does this Markov chain have a limiting distribution? (c) Consider a general three-state Markov chain with transition matrix P11 P12 P13 P = P21 P22 P23 P31 P32 P33 Give an example of a specific set of probabilities p;; for which the Markov chain is not irreducible (there is no single right answer to this, of course !).Problem 3. Consider the Markov chain shown in Figure 2. Figure 2: Problem 3 Markov chain 1. Let the initial distribution be Pr(A) : Pr(B) = 0.5. What is the probability distribution after one step? 2. What is the stationary distribution of the Markov chain? 5. A Markov chain {X,, n 2 0} with state space { 1, 2, 3, 4, 5 } has the transition probability matrix given below. DO P = Ooouo ooouo OHOOO HOO OOH (a) Draw the transition diagram for this Markov chain. (b) Is this Markov chain irreducible? Explain. (c) Determine the period of each state in this Markov chain.A stationary distribution of an m-state Markov chain is a probability vector q such that = q P, where P is the probability transition matrix. A Markov chain can have more than one stationary distribution. Identify all the stationary distributions that you can, for the 3-state Markov chain with transition probability matrix O O P Owl Does this Markov chain have a steady-state probability distribution ? 15 points