Let X1 denote the initial state at time 1 of the Markov chain for which the transition matrix is as specified in Exercise 5, and suppose that the initial probabilities are as follows:
Pr(X1 = 1) = 1/8, Pr(X1 = 2) = 1/4,
Pr(X1 = 3) = 3/8, Pr(X1 = 4) = 1/4.
Determine the probabilities that the chain will be in states 1, 2, 3, and 4 at time n for each of the following values of n:
(a) n = 2.
(b) n = 3.
(c) n = 4.