Please give it solution.

(10 points) Let (Xn)n=0,l,2... be a Markov chain with state space 8 = {1, 2, 3, 4} and transition probability matrix 0 l 0 0 _ 1/2 0 1/2 0 P O p 0 lp Wherepe[0,1]. 0 0 1/2 1/2 (a) Provide the transition graph. (b) Depending on the value p E [0, 1] determine all communication classes. (C) (d) Let p = 1. The initial distribution is given by aT = (1/4,1/2,1/4,0). Compute the distribution of all X" (n = 0, 1, . . .). Let p = 0. Determine all recurrent and all transient states