Question $2(5$ points$)$

Consider Stochastic Inventory Model given on Exercises of Markov Decision Processes and its solution is provided on Backward Induction Method. Furthermore, consider the same model parameters and assumptions, including demand distribution, function of revenue, holding cost and ordering cost which are all provided in the presentation: $2$ Case Study Markov Decision Processes.pptx$.$Download $2$ Case Study Markov Decision Processes.pptx$.$

Recall one of the assumption of this problem: $"$If demand exceeds inventory, the customer goes elsewhere and demand is lost". This question relaxes this assumption by introducing understocking case that we covered in Newsvendor Problem II $($Advanced$).$ That is$,$ when demand exceeds inventory, the demand is lost but this lost incurs c$($u$)$ understocking cost, where u is the amount of units that are understocked.

Assume that c$($u$)=$ u$/2$ and calculate the expected rewards in order to set up the Markov Decision Process.