A particle moves on a circle through points that have been marked 0, 1, 2, 3, 4 (in a clockwise order). The particle starts at point 0. At each step it has probability 0.5 of moving one point clockwise (0 follows 4) and 0.5 of moving one point counterclockwise. Let Xn (n > 0) denote its location on the circle after step n. {Xn} is a Markov chain.
(a) Construct the (one-step) transition matrix.
(b) Use your IOR Tutorial to determine the n-step transition matrix P(n) for n = 5, 10, 20, 40, 80.
(c) Use your IOR Tutorial to determine the steady-state probabilities of the state of the Markov chain. Describe how the probabilities in the n-step transition matrices obtained in part (b) compare to these steady-state probabilities as n grows large.