Let X = {Xn, n 0} be a Markov chain with state space E = {r, w, b, y} and transition matrix

Let X = {an 2 0} be a Markov chain with state space E = {1310, b,y} and transition matrix 0 0 1 0 _ 0 0.4 0.6 0 P _ 0.8 0 0.2 0 0.2 0.3 0 0.5 (a) Compute IP[X5 = b, X5 = r, X; = b, X3 = b| X4 = w]. (b) Compute ]E[f(X5)f(X5)|X4 = y] for the function f with values 2, 4. 7. and 3 at r, w, b, and g, respectively. Verify your answer with simulation