Reconsider the Cashco problem (see below ): formulate a Markov decision process and solve it to find the optimal decision rule that minimizes the average cost per period. Give the optimal rule µ ∗ and its average cost R ∗ . (At your option, use linear programming or dynamic programming).

Cashco problem: During any month, Cashco has a .5 chance of receiving a $1,000 cash inflow and a .5 chance that there will be a $1,000 cash outflow. For every $1,000 in cash on hand at the end of a month, Cashco incurs a $15 cost (due to lost interest). At the beginning of each month, Cashco can adjust its on-hand cash balance upward or downward with the cost per transaction being $20. Cashco can never let the on-hand balance become negative. The company is considering the following two cash management policies: Policy 1 At the beginning of a month in which the on[1]hand cash balance is $3,000, immediately reduce the cash balance to $1,000. At the beginning of a month in which the on-hand cash balance is $0, immediately bring the on-hand cash balance up to $1,000. Policy 2 At the beginning of a month in which the one-hand cash balance is $4,000, immediately reduce the cash balance to $2,000. At the beginning of a month in which the on-hand cash balance is $0, immediately bring the on-hand cash balance up to $2,000.