4.2.7 Consider a machine whose condition at any time can be observed and classified as being in one of the following three states:

State 1: Good operating order State 2: Deteriorated operating order State 3: In repair We observe the condition of the machine at the end of each period in a sequence of periods. Let Xn denote the condition of the machine at the end of period n for n D 1;2; : : : : Let X0 be the condition of the machine at the start. We assume that the sequence of machine conditions is a Markov chain with transition probabilities

and that the process starts in state X0 D 1.

(a) Find PrfX4 D 1g.

(b) Calculate the limiting distribution.

(c) What is the long run rate of repairs per unit time?

P11 0.9, P12=0.1, P130, P21 = 0, P22 0.9, P23 = 0.1, P31 = 1, P32 = 0, P33 = 0,