Show that, for an ergodic Markov chain (see Theorem 11.16), The second expression above shows that the

Question:

Show that, for an ergodic Markov chain (see Theorem 11.16),

The second expression above shows that the number K is independent of i. The number K is called Kemeny€™s constant. A prize was offered to the first person to give an intuitively plausible reason for the above sum to be independent of i. (See also Exercise 24.)