# Question

When a tennis player serves, he gets two chances to serve in bounds. If he fails to do so twice, he loses the point. If he attempts to serve an ace, he serves in bounds with probability 3/8. If he serves a lob, he serves in bounds with probability 7/8. If he serves an ace in bounds, he wins the point with probability 2/3.With an inbounds lob, he wins the point with probability 1/3. If the cost is 1 for each point lost and –1 for each point won, the problem is to determine the optimal serving strategy to minimize the (long-run) expected average cost per point.

(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (long-run) expected average cost per point in terms of the unknown steady-state probabilities (π0 , π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtainedin part (b) to find the optimal policy by exhaustive enumeration.

(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.

(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (long-run) expected average cost per point in terms of the unknown steady-state probabilities (π0 , π1, . . . , πM).

(c) Use your IOR Tutorial to find these steady-state probabilities for each policy. Then evaluate the expression obtainedin part (b) to find the optimal policy by exhaustive enumeration.

## Answer to relevant Questions

Each year Ms. Fontanez has the chance to invest in two different no-load mutual funds: the Go-Go Fund or the Go-Slow Mutual Fund. At the end of each year, Ms. Fontanez liquidates her holdings, takes her profits, and then ...Reconsider Prob. 19.2-3. (a) Formulate a linear programming model for finding an optimal policy. A student is concerned about her car and does not like dents. When she drives to school, she has a choice of parking it on the street in one space, parking it on the street and taking up two spaces, or parking in the lot. If ...Obtaining uniform random numbers as instructed at the beginning of the Problems section, use the acceptance-rejection method to generate three random observations from the probability density function The game of craps requires the player to throw two dice one or more times until a decision has been reached as to whether he (or she) wins or loses. He wins if the first throw results in a sum of 7 or 11 or, alternatively, ...Post your question

0