Partly solved answer is acceptable, thank you!

(Replacement Policy) A system is inspected at equally spaced points in time. After each inspection it is classied into one of L + 1 possible states {0,1,2...,L}. A system is in state 0 if it is found to be in the best possible condition. A system in state L is inoperative, while a system in state L 1 is in the worst possible condition but still able to operate. At every time period, the system state is likely to degrade by one unit with probability p. a. Let Xn denote the state of the system at time n. Find the transition probability matrix P. Consider now the following replacement policy; given 1* (O 1* then the system is replaced by a new one. Let X; denote the state of the system at time 77. under this policy. Find the transition probability matrix P". b. Are these chains ergodic ? If so, compute the stationary probability distribution