Let us revisit Exercise 2.

(a) Explain why the limiting distribution does not exist.

(b) Nevertheless, we can figure out the explicit pattern of the behavior of X_t. Using Excel or another software, for n = 4, compute P^t for, say, t = 2, ..., 8, and comment on the results. (P is a 5 × 5-matrix.) Check that a stationary distribution does exist: it is π = (0.0625, 0.25, 0.375, 0.25, 0.0625). Looking at the calculations you have already done, make sure that the average of two neighbor matrices, 1/2(P^2k−1 + P^2k), is approaching the matrix with all rows equal to π above.

(c) Check that a stationary distribution des exist and it is binomial: πi = (n i)²⁻ⁿ for (i) n = 4; (ii) for an arbitrary n.

Exercise 2

The Ehrenfest ball-box model. Consider a fixed number n of balls distributed somehow in two boxes. One at a time, a ball is selected at random from the total of n balls and moved from its box to the other. Let X_t be the number of balls in, say, the first box at step t.
Originally, this model was proposed as a model of heat exchange between two bodies isolated from the outside. At each step, an energy or temperature unit (a ball) is “transferred” from one body to the other. Another example may concern electrons independently occupying two orbits. Suppose that from time to time, one electron moves from its orbit to the other (due to receiving or emitting energy), and all electrons are equally likely to switch the orbits at the next step. Show that X_t is a Markov chain, and describe its transition matrix.