$25\mathrm{.}4$ Practice with the definitions

Consider each of the two simple DTMCs shown in Figure $25\mathrm{.}14.$ For each chain, please answer the following questions. Justify your answers by citing theorems.

$($a$)$ Is the chain aperiodic?

$($b$)$ Is the chain irreducible?

$($c$)$ Is the chain ergodic?

$($d$)$ Does the limiting distribution exist? If so$,$ what is it$?$ If not, why not?

$($e$)$ Does one or more stationary distributions exist? If so$,$ what are the stationary distribution$($s$)?$

$($f$)$ Is $p_c,$ the time$-$average fraction of time spent in state $c,$ well defined? If so$,$ what is it$?$

$($g$)$ Consider $m_{cc},$ the mean time until we again visit state $c,$ given we are in state $c.$ Is $m_{cc}$ well$-$defined? If so$,$ what is it$?$

Chain $1$

Chain $2$

Figure $25\mathrm{.}14$ Chains for Exercise $25\mathrm{.}4.$