Q1. A particle moves on a circle through points that have been marked 0, 1, 2, 3, 4 (points are marked in a clockwise order). The dynamics of the particle's movement is as follows:

The particle must move either clockwise or counter-clockwise at each step. It has a probability 0.6 of moving one point clockwise (0 follows 4) if the earlier move was clockwise. Similarly, it has the probability 0.7 of moving one point counter-clockwise (4 follows 0) if the earlier move was counter-clockwise.

(a) Can you model this movement as a Markov chain? Please specify the sequence of random variables {Xn}. Also specify transition probabilities in the form of a one-step transition matrix.

(b) Determine the n-step transition probabilities for n = 5, 10, 20, 40, 80.