Consider there Markov chain

Let P be the transition matrix of a Markov chain with 7 states. Which one of the following statements is not always true? O p2 is the transition matrix of a Markov chain with 71 states. If O is another transition matrix of a Markov chain with 72 states, then PQ) is the transition matrix of a Markov chain with 71 states. O If ) is another transition matrix of a Markov chain with 72 states, then *(P + Q) is the transition matrix of a Markov chain with 7, states. If P is invertible, then p-1 is the transition matrix of a Markov chain with 71 states.4. Consider the Markov chain X = {X,} with state space S = {0, 1, 2, ...} and transition probabilities 1 ifj=i-1 Puj =1 0 otherwise , for i 2 1 and Poo = 0, Poj = for j > 1. (a) Is this Markov chain irreducible? Determine the period for every state. (b) Is the Markov chain recurrent or transient? Explain. (c) Is the Markov chain positive recurrent? If so, compute the sta- tionary probability distribution. (d) For each state i, what is the expected number of steps to return to state i if the Markov chain X starts at state i? 5. Consider a Markov chain X = {X,} with state space S = {0, 1. 2,...} and transition probability matrix 0 1 0 O 0 0 P 0 4 0 0 P = O p 0 q 0 0 . . . 0 0 P 0 4 0 Here p > 0, q > 0 and p+q = 1. Determine when the chain is positive recurrent and compute its stationary distribution.1. For each of the following transition matrices, do the following: (1) Determine whether the Markov chain is periodic; (2) Give the period of each state; (3) Without using any software package, find /100 approximately (hint: you have computed stationary distribution of this two DTMC in HW5 and you can use that results); and (4) Identify if each state is recurrent, positive recurrent, or transient. (a) 0 1/3 2/3 P = 2/3 0 1/3 1/3 2/3 0 (b) .8 P = 5