4. (20 points) A certain stochastic process is modeled by a Markov chain that uses the transition matrix below. 0.2 0.6 0.1 0.1 0.05 0.25 0.4 0.3 P = 0.7 0.05 0.25 0 0 0.6 0 0.4 The state variable is Xt, and the state space is S = {1, 2, 3, 4}. Determine the following: (a) What is the matrix of expected visits in 1 epochs E(1)? (b) If P{Xo = 1} = 6, P{Xo = 2} = 3, and P{Xo = 3} = 5, what is p (1)? (c) What is mean sojourn time for state 4, E[N4]? (d) What is the probability that starting from state 1, first return time to state 1 is 1 epoch, $11