I need help with the following problems.Working solutions are acceptable.

Problem 1. Consider an irreducible nite Markov chain with states [1,1, ..., N. (a) (10| pts) Starting in state i, what is the probability the process will ever visit state 3'? (b) [10 pts] Let cl = ll'{visit state N before state Ulstart in 3'}. Find a set of linear equations which the r, satisfy, i: 0,1,...,N. (c) (10 pts) If 2:, jPM = 17 fort = 1, m, N 1, show that 3,: = MN is a solution to the equations in part (b). Problem 2. (25 points) A transition probability matrix P is said to be doubly stochastic if the sum over each column equals one, i.e. Ei'PiJ = 1 W H such a chain is irreducible and aperiodic and consists of M+1 states 0, 1, . . . , M, show that the stationary distribution is given by TFjZMH j=0,1,.n,M. Problem 3. (25 points) Let YR be the sum of n independent rolls of a fair die. Find limrW P{Yn is a multiple of 9}. (Hint: Apply the results of Problem 2}. Problem 4. (20 points) We have a gambler's ruin model. Simple random walk on states {0, 1, . . . , 10} with absorption at end points [1 and 10. {Recall that simple means that the probabilities of jumping left and right are equal: 10 = q = 1f2.) The process starts in state 6. What is the probability that the process will 1\"visit\" state 3 before visiting state 8