Let {X_n}_{n > 0} be an irreducible and positive recurrent Markov chain, and let its expected first return time to i, starting from i. Also, let e_i,j denote the expected number of times the chain, starting from i, visits j starting from time point 0, till before it makes its first return to i. Show that:

 

Transcribed Image Text:

eij 馬 eij 馬

Answer rating: 100% (QA)

To show that eij ij we will use the fact that for an irreducible and positive recurrent Markov chain the limiting probabilities i ie the longterm prop... View the full answer