At 9:00 AM every day an independent financial advisor will see a client who is concerned about retirement. A client has one of the following three types of concerns: creating a new retirement plan, managing investments under an existing retirement plan, or generating income during retirement. The retirement concerns of the clients form a Markov chain. The states are the clients’ three retirement concerns. Let the state X_n denote the retirement concern of the client who is seen on day n. The financial advisor’s historical records indicate that if today’s client is interested in creating a new retirement plan, the probabilities that tomorrow’s client will be interested in managing investments under an existing retirement plan, or generating income during retirement, are 0.2 and 0.5, respectively. If a client today is interested in managing investments under an existing retirement plan, the probabilities that a client tomorrow will be interested in creating a new retirement plan, or generating income during retirement, are 0.4 and 0.1, respectively. Finally, if today’s client is interested in generating income during retirement, the probabilities that tomorrow’s client will be interested in creating a new retirement plan, or managing investments under an existing retirement plan, are 0.3 and 0.6, respectively. The financial advisor charges the following fixed fees each time that a client visits. The fees for creating a new retirement plan, managing investments under an existing retirement plan, and generating income during retirement are $500, $300, and $400, respectively.

(a) Model the fi nancial advisor’s daily 9:00 AM meetings with clients as a three-state recurrent MCR. Construct the transition probability matrix and the reward vector.

(b) Execute value iteration to find the vector of expected total rewards that the financial advisor will earn from her retirement plan clients after 3 days. Assume zero terminal rewards at the end of day 3. 

(c) Find the financial advisor’s expected average reward, or gain.

(d) Find the financial advisor’s vector of expected total discounted rewards using a discount factor of α = 0.9.