Help. Thank you

3. a. Construct a Markov chain with FOUR states { A, B, C, D } whose long-term behaviour, as time t - oo tends to infinity, is such that Poo(A - A) = 0.15, Po(A - B) = 0.35, Poo(A - C) = 0.23, Poo(A - D) = 0.27 State the transition matrix and draw the diagram of the resulting Markov chain. [13 marks] b. What is the most unlikely possible path from B to C of length less than (or equal to) 4 in your proposed Markov chain? [4 marks] c. Modify your Markov chain by introducing a fifth state E such that p.(E - E) = 0 but px(E - E) # 0, for all k > 0. [8 marks]7. (Markov Chain) Let X,, be the Bernoulli iid random process, and let Y,, be given by Yn = Xn + Xn-1 1) Determine whether Y, is a Markov process or not and, if it is, find the transition diagram. Consider the vector process defined by Zn = (X,, Xn-1) 2) Determine whether Z, is a Markov process or not and, if it is, find the transition diagram. 3) Find the transition matrix Q of Z.. 4) Compute Q". 5) Find the steady state probabilities for Zn