1. Consider a Markov chain (Xn)n0 whose state space is the set AXES of pairs of integers (i,j) such that either i = 0 or j = 0. Starting at (0, 0), the first step is to either (0, 0) or (1, 0) or (1, 0) or (0, 1) or (0, 1), each with probability 1/5. Starting from any other (i,j) AXES, the first step is to one of the two neighboring points in AXES obtained by incrementing either i or j by 1 or -1, moving with probability 1 p one step closer to (0, 0), and with probability p one step away from (0, 0) (assume p (0, 1)).
2. (a) Under what condition on p is this Markov chain transient? Explain briefly.
3. (b) Find all p such that the Markov chain has a stationary distribution, find it, and explain why it is unique.
4. (c) Is this Markov chain aperiodic?
5. (d)Evaluate limn P(Xn = (0, 0)) for each value of p.
6. (e) What is the mean return time for state (0, 0) for each value of p?