Question: Markov Chains, I'm getting a bit stuck on this problem set Consider a Markov chain (X0, X1, .. . }, specied by the following transition
Markov Chains, I'm getting a bit stuck on this problem set


Consider a Markov chain (X0, X1, .. . }, specied by the following transition probability graph. 1.P(X2=2|X0=1)= 2. Find the steadystate probabilities 7n , 7:2. and 7:3 associated with states 1, 2. and 3. respectively. 07;] '752 07173 3. For n = 1, 2, let Y,. = X,' X,._1.Thus, Y,' = 1 indicates that the nth transition was to the right, Y,, = 0 indicates that it was a selftransition, and Y" = 1 indicates that it was a transition to the left. ljm P(Y,, = 1) = n>no 4. Is the sequence Y1, Y2, a Markov chain? No V 5. Given that the nth transition was a transition to the right (Y,l = 1), nd (approximately) the probability that the state at time n 1 was state 1 (Le, X,._1 = 1). Assume that n is large. 6. Suppose that X0 = 1. Let T be the rst positive time index n at which the state is equal to 1. 1.P(X2=2|X0=1)= 2. Find the steady-state probabilities 7:1, 7:2, and 71:3 associated with states 1, 2, and 3, respectively. .\"1 .753 3. For n = 1, 2, let Yn = Xn X,,_1. Thus, Y,, = 1 indicates that the nth transition was to the right, Yn = 0 indicates that it was a self-transition, and Y" = 1 indicates that it was a transition to the left. lim P(Yn = 1) = nNXJ 4. Is the sequence Y1, Y2, a Markov chain? No v 5. Given that the nth transition was a transition to the right (Yn = 1), find (approximately) the probability that the state at time n 1 was state 1 (Le, X -1 = 1). Assume thatn is large. 6. Suppose that X0 = 1. Let T be the rst positive time index n at which the state is equal to l. E [T] = 7. Does the sequence X1, X2, X3, converge in probability to a constant? No v 8. Let Z, = max{X1, , X,,}. Does the sequence 21 , 22, 23, converge in probability to a constant? No v
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
