Consider the following Markov chain on {1,..., N} defined by the following transition diagram P core...

  

Transcribed Image Text:

Consider the following Markov chain on {1,..., N} defined by the following transition diagram P core retos COF a. Write the transition matrix for N = 4. b. We suppose p, q> 0. What are the communicating classes (briefly justify your answer)? c. We suppose p, q> 0 and r > 0. What is the period of each state (briefly justify)? d. We suppose p, q> 0 and r = 0. If n is odd, show that the chain is aperiodic. = e. (bonus) We suppose p, q > 0 and r = 0. If N is even, show that the chain has period 2. f. Suppose N = 4, p = 0.3, and q 0.2 and that N is now an absorbing state. Complete and run the following cell from week 2's jupyter notebook (available at this link), to get the mean time spent in 3 starting from 2. (you can either handwrite/type the answer or include a screenshot). Starting from 2, what is the probability that the chain never returns to 2? g. (bonus) Let's consider the original Markov chain, and suppose p, q> 0 and r = = 0, with p0.5. What is the probability to visit all the other states before returning to its initial position (hint: assuming that the chain starts at 1, you can condition on the first event and use results Consider the following Markov chain on {1,..., N} defined by the following transition diagram P core retos COF a. Write the transition matrix for N = 4. b. We suppose p, q> 0. What are the communicating classes (briefly justify your answer)? c. We suppose p, q> 0 and r > 0. What is the period of each state (briefly justify)? d. We suppose p, q> 0 and r = 0. If n is odd, show that the chain is aperiodic. = e. (bonus) We suppose p, q > 0 and r = 0. If N is even, show that the chain has period 2. f. Suppose N = 4, p = 0.3, and q 0.2 and that N is now an absorbing state. Complete and run the following cell from week 2's jupyter notebook (available at this link), to get the mean time spent in 3 starting from 2. (you can either handwrite/type the answer or include a screenshot). Starting from 2, what is the probability that the chain never returns to 2? g. (bonus) Let's consider the original Markov chain, and suppose p, q> 0 and r = = 0, with p0.5. What is the probability to visit all the other states before returning to its initial position (hint: assuming that the chain starts at 1, you can condition on the first event and use results