1. A machine can be in four different statm: new, good, bad, and dead. 1When the machine is in new condition, there is a prot of $10,000. 'When the machine is in good condition, there is a prot of $5,000. 'When the machine is in bad condition, there is a cost of $1,000. When the machine is dead, there is a cost of $5,000. It is possible to repair or replace a machine when it is in good, bad, or dead state. The replacement cost is $10,000 and the next day, we have a brand new machine. Repairing a good machine costs $1000 and returns it to state new with probability 0.? but with probability 0.3 the repair may he unsuccessful and the machine stays in the same state. On the other hand, repairing a bad machine costs $2,000 and repairing a dead machine costs $4,000. Repairing a bad machine returns it to good condition the next day. Repairing a dead machine returns it to good with probability 0.0 and had with probability 0.4. If you leave the machine to itself, the state of the machine for states new, good, bad1 and dead evolves with respect to a Markov chain with probability transition matrix 0.8 0.2 0.0 0.0 0.0 0.5 0.4 0.0 0.0 0.0 0.5 0.5 0.0 0.0 0.0 1.0 Suppose that the objective is to maximize the total expected prot over the next 20 days assuming that the terminal reward for each state is the corresponding daily prot. - Formulate this as a nite horizon MDP. I Suppose that at each decision epoch you use the following decision rule: you repair a had machine and replace a dead machine and leave the machine to itself in other states. \"01st is the expected total prot over 30 days under this policy for each possible initial state? 1 I 1What is the optimal policy and the corresponding optimal expected total prot over 230 days for each initial state?I