Section 2: Markov Decision Processes Question 1: What is the special property of Markov chains? Question 2: Everyday morning, the working condition of an equipment is inspected, then classified as: = state 0 = as new, state 1 = slight damage, state 2 important damage, or state 3 = out of order. = Suppose the manager has two operation policies to choose from and applies the first one defined as follows: Policy 1: do nothing in states 0, 1, 2 and replace the equipment in state 3. Suppose that the replacement process takes one day to complete with a lost profit of $1000 and a replacement cost of $ 500, and that the costs of defectives are $ 0 for state 0, $ 500 for state 1, and $ 800 for state 2. If the steady-state probabilities of Policy 1 are: 10=2/9, A1=2/9, 12=3/9, 13=2/9 1. What is the expected average operation cost per day? Explain your answer 2. If the expected average operation cost per day for policy 2 is $ 666.66, what is the optimal policy? 3. Suppose that the manager applies an operation policy with the following transition matrix, What is the steady-state probabilities of the operation policy? Explain your answer 0 0 1 2 3 0 0 1/2 1/20 1 1 0 1/2 1/4 1/4 2 0 0 1/2 1/2 0 0 = 3 1 0