(a) Four blue chips and four white chips are distributed in two boxes, with each of them containing four chips. At each step, we draw one chip uniformly at random from each of the two boxes and then exchange the places of the two selected chips (so that the chip drawn from the rst box is put to the second and vice versa). If Xt denotes the number of blue chips in the rst box after t steps, determine the transition matrix for the Markov chain {Xt }t0. (b) Suppose that transition probabilities of a Markov chain with state space S = {1, 2, 3, 4, 5, 6} satisfy p (4)(1, 1) = 0.04. Show that p(1, 2)p(2, 1) 0.2. (c) Give an example of a Markov chain on S = {1, 2, 3, 4}, such that there are exactly two communi- cating classed and 1 is the only recurrent state.