Problem $8$

$4\mathrm{.}0$ points possible $($graded$,$ results hidden$)$

Bob's phone plan has a voicemail service with the storage capacity of maximum $2$ voice messages.

Each morning, Bob checks and answers the voice messages according to his available time during that morning. The maximum number of messages that Bob has time to answer each morning is with probability $,$ with probability $,$ with probability $.$ Bob will answer as many messages as he has time for each morning, e$.$g$.$ if there is only $1$ message in the voice mailbox, but he has time to answer $2$ messages that morning, then he will answer that $1$ message. After Bob answers a message, the message is removed from the voice mailbox.

The number of messages that Bob receives the day before is with probability $,$ with probability $,$ and $2$ with probability $.$ If the voice mailbox is already full, all newly arrived messages will be discarded. For simplicity, assume that there is no voice messages received in the mornings during the time he checks.

A Markov chain model:

The above system can be modeled by a discrete$-$time homogeneous Markov chain as in the figure below: