# Question

Every Saturday night a man plays poker at his home with the same group of friends. If he provides refreshments for the group (at an expected cost of $14) on any given Saturday night, the group will begin the following Saturday night in a good mood with probability 7/8 and in a bad mood with probability1/8. However, if he fails to provide refreshments, the group will begin the following Saturday night in a good mood with probability 1/8 and in a bad mood with probability 7/8, regardless of their mood this Saturday. Furthermore, if the group begins the night in a bad mood and then he fails to provide refreshments, the group will gang up on him so that he incurs expected poker losses of $75. Under other circumstances, he averages no gain or loss on his poker play. The man wishes to find the policy regarding when to provide refreshments that will minimize his (long-run) expected average cost per week.

(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (longrun) expected average cost per period in terms of the unknown steady-state probabilities (π0, π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.

(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (longrun) expected average cost per period in terms of the unknown steady-state probabilities (π0, π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.

## Answer to relevant Questions

When a tennis player serves, he gets two chances to serve in bounds. If he fails to do so twice, he loses the point. If he attempts to serve an ace, he serves in bounds with probability 3/8. If he serves a lob, he serves in ...Read the referenced article that fully describes the OR study summarized in the application vignette presented in Sec. 19.2. Briefly describe how Markov decision processes were applied in this study. Then list the various ...During any period, a potential customer arrives at a certain facility with probability 1 2. If there are already two people at the facility (including the one being served), the potential customer leaves the facility ...Obtaining uniform random numbers as instructed at the beginning of the Problems section, use the acceptance-rejection method to generate three random observations from the triangular distribution used to illustrate this ...Each time an unbiased coin is flipped three times, the probability of getting 0, 1, 2, and 3 heads is 1/8, 3/8, 3/8, and 1/8, respectively. Therefore, with eight groups of three flips each, on the average, one group will ...Post your question

0