Hi, please can you answer this Stochastic Processes question for me, thank you.

4. (a) Boy Tom is always in one of three states: eat, play and sleep. He eats on average for 20 minutes at a time; plays on average for 1.5 hours; and sleeps on average for 4 hours. After eating, he has a 30% chance that he will sleep and a 70% chance that he will play. After playing, there is a 80% chance that he will eat and a 20% chance that he will sleep. And after sleeping, he always plays. (i) Let X (t) denote Tom's state at time t. Assuming that (X (t) : t 2 0) is a continuous time Markov chain, write down the generator matrix. [3 marks] (ii) Show that the above continuoustime Markov chain is positive recurrent. [3 marks] (iii) Calculate its stationary distribution. [5 marks] (iv) Let p15 (:5) denote the probability that a process which starts in state i at time 0 is in state j at time t, where 2', j E S 2 {eat, play, sleep}. Write down the Kolmogorov Forward Equation for pj (t) and identify the solutions for pi(t),i E S. [7 marks] (b) Consider a time-homogeneous Markov chain (Xn : n = 0, 1,2,...) with state space S = {1, 2, 3, 4, 5, 6} and transition probability matrix 0 (J 1 U 0 0 0 0 0 0 1/2 1/2 0 o 0 1/2 1/2 0 1/8 3/8 0 1/2 0 o 1 0 0 0 0 0 0 1/3 0 0 0 2/3 (i) Sketch a transition diagram for the above chain. [2 marks] (ii) Calculate the conditional probabilities lP'(X3 = 3|X1 = 1,X0 = 1) and MK; = 2,X2 = 1]X1 = 1). [6 marks] (iii) Calculate the following conditional expectations E[X1|X0 = 4), spit/pmJ = 4). [4 marks] (iv) Find the communication classes for the chain and determine whether they are closed or not. [3 marks] (v) Determine whether the chain is irreducible or not. [3 marks] (vi) For each state j E S, show whether it is recurrent or transient. [4 marks]