6 This problem will show why steady-state probabilities are sometimes referred to as stationary probabilities. Let p1, p2, . . . , ps be the steady-state probabilities for an ergodic chain with transition matrix P. Also suppose that with probability pi, the Markov chain begins in state i.

a What is the probability that after one transition, the system will be in state i? (Hint: Use Equation (8).)

b For any value of n(n  1, 2, . . .), what is the probability that a Markov chain will be in state i after n transitions?

c Why are steady-state probabilities sometimes called stationary probabilities?