The condition of a machine, which is observed at the beginning of each day, can be represented by one of the following four states:

The next state of the machine depends only on the present state and is independent of its past history. Hence, the condition of the machine can be modeled as a four-state Markov chain. Let X_n−1 denote the state of the machine when it is observed at the start of day n. The state space is E = {1, 2, 3, 4}. Assume that at the start of each day, the engineer in charge of the machine does nothing to respond to the condition of the machine when it is in states 2, 3, or 4. A machine in states 2, 3, or 4 is allowed to fail and enter state 1, which represents a repair process. Therefore, a machine in state 1, not working, is assumed to be under repair.  

The repair process carried out when the machine is in state 1 (NW) takes 1 day to complete. When the machine is in state 1 (NW), the repair process will be completely successful with probability 0.5 and transfer the machine to state 4 (WP), or largely successful with probability 0.2 and transfer the machine to state 3 (mD), or marginally successful with probability 0.2 and transfer the machine to state 2 (MD), or unsuccessful with probability 0.1 and leave the machine in state 1 (NW). A machine is not repaired when it is in states 2, 3, or 4. A machine in state 2 (MD) will remain in state 2 with probability 0.7, or fail with probability 0.3 and enter state 1 (NW). A machine in state 3 (mD) will remain in state 3 with probability 0.3, or acquire a major defect with probability 0.5 and enter state 2, or fail with probability 0.2 and enter state 1 (NW). Finally, a machine in state 4 (WP) will remain in state 4 with probability 0.4, or acquire a minor defect with probability 0.3 and enter state 3, or acquire a major defect with probability 0.2 and enter state 2, or fail with probability 0.1 and enter state 1 (NW). Construct the transition probability matrix for this four-state Markov chain model under which a machine is left alone in states 2, 3, and 4, and repaired in state 1.

Transcribed Image Text:

State Description 1 (NW) 2 (MD) 3 (mD) Not Working, Under Repair Working, with a major defect Working, with a minor defect 4 (WP) Working Properly