Exercise $4\mathrm{.}2($Lawler$,$ Exercise $1\mathrm{.}9($d$)-($e$)$; $4$ points$).$ Recall the Markov chain on state space $\{1,2,3,4,5\}$

with transition matrix

P $=$

$$

$$

$$

$$

$$

$$

$01/32/300$

$0001/43/4$

$0001/21/2$

$10000$

$10000$

$$

$$

$$

$$

$$

$$

$1.$ Let T$1$ be the first return time to $1$ if we start the chain at $1.$ What is the distribution of T and what

is E$[$T $]?$ What does this say, without any further calculation, about $\backslash$pi $(1)$ where $\backslash$pi is the invariant

probability?

$2.$ Find the invariant probability $\backslash$pi $.$ Use this to find the expected return time to $2,$ starting in state $2$