1. (2 points) A particle moves on a circle through points that have been marked 1, 2, 3, 4 (in a clockwise order). The particle starts at point 1. At each step it has probability p of moving one point clockwise and 1 - p of moving one point counterclockwise. Find the two-step transition matrix when a. p = 0.5 b. p = 0.2 2. (2 points) Given the following (one-step) transition matrices of a Markov chain, determine the classes of the Markov chain and whether they are recurrent or transient. State 0 1 2 3 State 0 1 2 3 31 1 0 0 0 00 2 0 LO 1 0 0 00 0 P = 1 P= 0 1 2 3 1 0 0 0 0 1 0 0 0 3 3. (2 points) I have 5 umbrellas, some at home and some in the office. I keep moving between home and office. I take an umbrella with me only if it rains. If it does not rain, I leave the umbrella behind (at home or in the office). It may happen that all umbrellas are in one place, I am at the other, and it starts raining and must leave, so I get wet. a. If the probability of rain is 0.3, what is the probability that I get wet? b. How many umbrellas should I have so that, if I follow the strategy above, the probability get wet is less than 2%? To solve the problem, consider a Markov chain taking values in the set S = {0,1,2,3,4,5), where each state represents the number of umbrellas in the place where I am currently at (home or office). If i = 1 and it rains then I take the umbrella, move to the other place, where there are already 4 umbrellas, and, including the one I bring, I will have 5 umbrellas. So the state transits from 1 to 5 with probability of raining which is Pas = 0.3. Now assume that i = 1 but it is not raining, so I move to the other place without an umbrella and I have 4 umbrellas over there. Thus, P14 = 0.7. With the same procedure for finding other transition probabilities, we have