Deduce the invariant distribution. (Oxford 2005)

* 14. Consider a Markov chain with state space S = {0, 1, 2, . . . } and transition matrix given by pi, j =

(

qp j−i+1 for i ≥ 1 and j ≥ i − 1, qp j for i = 0 and j ≥ 0, and pi, j = 0 otherwise, where 0 < p = 1 − q < 1.

For each p ∈ (0, 1), determine whether the chain is transient, null recurrent, or positive recurrent, and in the last case find the invariant distribution. (Cambridge 2007)

15. Let (Xn : n ≥ 0) be a simple random walk on the integers: the random variables ξn := Xn − Xn−1 are independent, with distribution P(ξ = 1) = p, P(ξ = −1) = q, where 0 < p < 1 and q = 1 − p. Consider the hitting time τ = inf{n : Xn = 0 or Xn = N}, where N > 1 is a given integer. For fixed s ∈ (0, 1), define Hk = E

????

sτ 1(Xτ = 0)

X0 = k

for k = 0, 1, . . . , N.