# Question: During any period a potential customer arrives at a certain

During any period, a potential customer arrives at a certain facility with probability 1 2. If there are already two people at the facility (including the one being served), the potential customer leaves the facility immediately and never returns. However, if there is one person or less, he enters the facility and becomes an actual customer. The manager of the facility has two types of service configurations available. At the beginning of each period, a decision must be made on which configuration to use. If she uses her “slow” configuration at a cost of $3 and any customers are present during the period, one customer will be served and leave the facility with probability 3/5. If she uses her “fast” configuration at a cost of $9 and any customers are present during the period, one customer will be served and leave the facility with probability 4/5. The probability of more than one customer arriving or more than one customer being served in a period is zero. A profit of $50 is earned when a customer is served.

(a) Formulate the problem of choosing the service configuration period by period as a Markov decision process. Identify the states and decisions. For each combination of state and decision, find the expected net immediate cost (subtracting any profit from serving a customer) incurred during that period.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (long-run) expected average net cost per period in terms of the unknown steady-state probabilities (π0, π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.

(a) Formulate the problem of choosing the service configuration period by period as a Markov decision process. Identify the states and decisions. For each combination of state and decision, find the expected net immediate cost (subtracting any profit from serving a customer) incurred during that period.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (long-run) expected average net cost per period in terms of the unknown steady-state probabilities (π0, π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.

## Answer to relevant Questions

A student is concerned about her car and does not like dents. When she drives to school, she has a choice of parking it on the street in one space, parking it on the street and taking up two spaces, or parking in the lot. If ...(a) The numbers x1, x2, . . . , xn and y1, y2, . . . , yn are random observations from the same exponential distribution. (b) The average of x1, x2, . . . , xn is equal to the average of y1, y2, . . . , yn. (c) z is a random ...The weather can be considered a stochastic system, because it evolves in a probabilistic manner from one day to the next. Suppose for a certain location that this probabilistic evolution satisfies the following ...Obtaining uniform random numbers as instructed at the beginning of the Problems section, generate four random observations (approximately) from a normal distribution with mean = 0 and standard deviation = 1. (a) Do this by ...Reconsider Prob. 20.4-6 involving the game of craps. Now the objective is to estimate the probability of winning a play of this game. If the probability is greater than 0.5, you will want to go to Las Vegas to play the game ...Post your question