Consider the stochastic equipment replacement problem we discussed in class.

Specifically, we need to have a machine for $4$ periods. The quality of the machine $-$indexed with j$-$ can

be $0,1,$ or $2$ with $0$ being the best quality corresponding to a new machine. The cost of a new

machine is R$=5.$ The per$-$period cost of maintaining a machine of quality j is C$($j$)$ where C$(0)=2,$ C$(1)=4,$

C$(2)=6.$ The salvage value of a machine of any quality is zero. The transition probabilities are given by

the following table:F$.$ State

$0$

$01/3$

B$.$ State $10$

$1$

$1/3$

$1/5$

$2$

$1/3$

$4/5$

$2001$

Assume you start period $1$ with new machine. Find the optimal $($cost minimizing$)$ policy.

Please write a very short $``$report$$ where you first state the optimal policy and then you show your

calculations and any code you used $($if you used one$).$ The presentation of your report will be worth $5$ points

$($even your results are prefect, you can lose up to $5$ points you present them in a sloppy manner$).$