Topic: Markov Chains

Part 1

show complete calculations

1.4. A Markov chain X, X,, X2, . . . has the transition probability matrix 0 1 2 0 0.1 0.1 0.8 P = 1 0.2 0.2 0.6 2 0.3 0.3 0.4 Determine the conditional probabilities Pr(X, = 1, X, = 1|X, =0) and Pr(X, = 1, X, = 1/X, = 0). 1.5. A Markov chain Xo, X,, X2, . . . has the transition probability matrix 0 1 2 0 0.3 0.2 0.5 P = 1 0.5 0.1 0.4 2 10.5 0.2 0.3 and initial distribution p, = 0.5 and p, = 0.5. Determine the probabilities Pr(X, = 1, X, = 1, X, = 0) and Pr(X, = 1, X, = 1, X, = 0).\f1.4. The random variables ,, 5,, . . . are independent and with the com- mon probability mass function k = 0 2 3 Pr($ = k) = 0.1 0.3 0.2 0.4 Set Xo = 0, and let X,, = max (5,. . . .. 5,) be the largest & observed to date. Determine the transition probability matrix for the Markov chain (X,)