$1.$ Consider an inventory system in which the sequence of events during each period is as follows. $(1)$ We observe the inventory level $($call it i$)$ at the beginning of the period. $(2)$ If $\backslash (\backslash$mathrm$\{$i$\}\backslash$leq $1,4\backslash )-$ i units are ordered. If $\backslash (\backslash$mathrm$\{$i$\}\backslash$geq $2,0\backslash )$ units are ordered. Delivery of all ordered units is immediate. $(3)$ With probability $\backslash (1/4,0\backslash )$ units are demanded during the period; with probability $\backslash (1/4,1\backslash )$ unit is demanded during the period; with probability $\backslash (1/4,2\backslash )$ unit is demanded during the period; and with probability $\backslash (1/4,3\backslash )$ units are demanded during the period. $(4)$ We observe the inventory level at the beginning of the next period. Define a period's state to be the period's beginning inventory level.

$($a$)(6$ points$)$ Model this situation as a stochastic process.

$($b$)(6$ points$)$ What is the expected number of stockout?

$($c$)(6$ points$)$ What is the average inventory level?

$($d$)$ Suppose that we have $3$ units in the inventory at the present.

i$.(6$ points$)$ What is the expected number of periods you will have zero inventory before observing a stockout? $($Hint: Use Fundamental matrix$)$

ii$.(6$ points$)$ What is the probability that your will give an order before observing a stockout?

$($Hint: Use Fundamental matrix$)$