Problem 7.4 (10 points) A Markov chain Xo, X1, X2, ... with state space S = {1, 2,3, 4) has the following transition graph: 0.5 0.5 0.5 1 0.5 0.5 0.5 2 0.5 0.5 (a) Provide the transition matrix for the Markov chain. (b) Determine all recurrent and all transient states. (c) Determine all communication classes. Is the Markov chain irreducible? (d) Find the stationary distribution. (e) Can you say something about the limiting distribution of this Markov chain?Problem 3. Consider the Markov chain shown in Figure 2. Figure 2: Problem 3 Markov chain 1. Let the initial distribution be Pr(A) : Pr(B) = 0.5. What is the probability distribution after one step? 2. What is the stationary distribution of the Markov chain? 8. (10 points)(The Weak Law of Large Numbers) In order to estimate f, the true fraction of smokers in a large population, Alvin selects n people at random. His estimator M., is obtained by dividing S,, the number of smokers in his sample, by N, i.e., M. = S,. Alvin chooses the sample size n to be the smallest possible number for which the Chebyshev inequality yields a guarantee that P(IMn - f1 2 () So