Stochastic Processes, Markov chain

1. The random variables 8, $1, $2, ... are independent and identically distributed with distribution P($ = 0) = 1/4 and P($ = j) = c/j for j = 1, 2,3. Let Xo = 0 and Xn = max($1, . . ., En) for n = 1, 2, .... (a) What value must c take? (b) Explain why {Xn, n = 0, 1, 2, ...} is a Markov chain. (c) Write down the transition matrix. (d) Draw the transition diagram and classify the states (aperiodic, transient, re- current, eorgodic, etc). (e) Calculate P(Xn = 0). (f) Calculate P(X4 = 3, X2 = 1/X1 = 3)