Here it is

Question 5. (18 pts) Let {Xn }n=0,1,... be an irreducible discrete-time Markov chain having state Space 8 and transition matrix P = {H,j},,,-E 3. Let p 6 (0, 1) be a constant. Construct a new stochastic process {YR },,=0,1,___ as follows: in each step, with probability p, the chain will move according to P. That is, assuming it is in state i, then it will move to state 3' with probability PM. With probability 1 p, the chain will do nothing and stay in the same state. The choice between these two options in each step is independent of everything else. It can be seen that {Yn}n=0,1,... is also an irreducible discrete-time Markov chain (you do not need to prove this). (a) (6 pts) Assume Y0 = 2'. What is E(min{n : l!"n 75 i}), the expected time that {Yn}n=o,1,... leaves state i? (b) (6 pts) Show that if {Xn}n=0,1,... is positive recurrent, then {Yn}n=0,1,,.. is also positive recur rent. (0) (6 pts) Now assume instead of a constant, the probability of following P can depend on the current state. That is, assuming the chain is currently in state 2', then for the next step, it will move according to P with probability pi, and stay in state 2' with probability 1 pi. Is the result that we proved in part (b) still true? If yes, explain why; if no, give a counterexample