S Ergodic theorem for Markov chains in continuous time. In the situation of Problem 6.28, suppose that E is finite and G irreducible, in that for all x; y 2 E there is a k 2 N and x0; : : : ; xk 2 E such that x0 D x, xk D y and Qk iD1 G.xi1; xi / ¤ 0. Show the following:

(a) For all x; y 2 E there is a k 2 ZC such that limt!0…t .x; y/=tk > 0. Consequently, for all t > 0, all entries of …t are positive.

(b) For all x; y 2 E, limt!1…t .x; y/ D ˛.y/, where ˛ is the unique probability density on E that satisfies one of the equivalent conditions ˛G D 0 or ˛…s D ˛ for all s > 0.