A woman wishes to sell a car within the next 4 weeks. She expects to receive one bid or offer each week from a prospective buyer. The weekly offer is a random variable, which has the following stationary probability distribution: 

Once the woman accepts an offer, the bidding stops. Her objective is to maximize her expected total income over a 4-week planning horizon.

(a) Formulate this optimal stopping problem as a unichain MDP.

(b) Use value iteration to find an optimal policy which will specify when to accept or reject an offer during each week of the planning horizon.

Transcribed Image Text:

Offer per week $16,000 $18,000 $22,000 $24,000 Probability 0.40 0.10 0.30 0.20