0.2 0.8 0.4 0 2. Consider a Markov chain with a finite state space S = {81, 82, 83, 84, modelling four different states that your insurance company can be in at the end of the fiscal year. We refer to s, as state i. In state 1, things are not so good and your company has made a loss for the year of 1 million pounds. In state 2, your company breaks even, earning precisely o pounds. In state 3, things are amazing, earning you 10 million pounds for the year. In state 4, things are also quite good with earnings of 4 million pounds for the year. The transitions between the states are modelled by the transition matrix 0.4 0.3 p(1,3) p(1,4) 0 0 P= 0.6 0 p(4,1) 0p(4,3) p(4.4) If you find it helpful, you are welcome to use drawings to justify your answers. However, it is also fine to just refer to the entries of the transition matrix. (a) (1 mark] Suppose p(1, 4) = 0 and p(4,1) = P(4, 4) = 0.4. What are then the values of p(1.3) and p(4,3)? (b) (5 marks] In the setting of (a), identify the closed communication classes, the transient states, and the recurrent states. Briefly explain your reasoning in deducing these clasifications. Is the Markov chain irreducible? (c) (5 marks] Now suppose instead that p(1,4) = 0.2 while p(4,1) = p(4,3) = 0. How do your answers to (b) change? Again, you should briefly explain how you arrive at the different clasifications. (d) [5 marks] Finally, consider P with p(1,4) = 0.2 and p(4,1) = p(4,4) = 0.4. Recalling that every finite Markov chain has a stationary distribution 7, you may use that 7 = (0.3, 0.55, 0.05, 0.1) is a stationary distribution for this transition matrix P. Write down what it means that this is a stationary distribution. Explain, by appealing to a theorem from lectures, why these probabilities can in fact be interpreted as the long-run probabilities of being in states 1, 2, 3, and 4, respectively (you need to briefly justify why the theorem can be applied). Explain whether or not the theorem applies in (b) and (c). (e) [4 marks] Let P be as in (d). In the long run, how often do you expect to be making 10 million pounds for the year, and what are the average yearly earnings that you expect of your insurance company