Suppose a Bernoulli trial results in a success with probability p and a failure with probability 1 – p. Suppose the Bernoulli trial is repeated indefinitely with each repitition independent of all others. Let Xn be a “ success runs” Markov chain where represents the number of most recent consecutive successes that have been observed at the n th trial. That is Xn = m, if trial numbers n, n– 1, n – 2,…, n– m+ 1 were all successes but trial number n–m was a failure. Note that Xn = 0 if the th trial was a failure.
(a) Find an expression for the one- step transition probabilities, p i, j.
(b) Find an expression for the - step first return probabilities for state 0,f0(n)0 .
(c) Prove that state 0 is recurrent for any 0 < p < 1. Note that since all states communicate with one another, this result together with the result of Exercise 9.29 is sufficient to show that all states are recurrent.