Need help with part b. Please use MATLAB

2 Markov Chains 4. Markov Chain Distributions. Consider a Markov Chain with the following transition graph. Page 2 of 3 MECH 6326 Optimal Control and Dynamic Programming Homework \#2 (Self-loops are omitted, since these transition probabilities can be computed from the other transition probabilities.) Suppose the initial state is x0=1, so that the initial distribution is (a) Use distribution propagation to compute the following quantities at time T=50 : - P(xT=1) T+11t=0TP(xt=1). (b) Then use Monte Carlo estimation to approximate these quantities. Generate plots of your estimates versus the number of sample trajectories used from 10 up to 10000 sample trajectories. Compare your Monte Carlo estimates to the exact values computed in part (a). 2 Markov Chains 4. Markov Chain Distributions. Consider a Markov Chain with the following transition graph. Page 2 of 3 MECH 6326 Optimal Control and Dynamic Programming Homework \#2 (Self-loops are omitted, since these transition probabilities can be computed from the other transition probabilities.) Suppose the initial state is x0=1, so that the initial distribution is (a) Use distribution propagation to compute the following quantities at time T=50 : - P(xT=1) T+11t=0TP(xt=1). (b) Then use Monte Carlo estimation to approximate these quantities. Generate plots of your estimates versus the number of sample trajectories used from 10 up to 10000 sample trajectories. Compare your Monte Carlo estimates to the exact values computed in part (a)