Consider an inventory problem as in Example 2, but with finite time horizon \(T=2\) and no discount factor. Let \(r\) remain as a parameter and work through the backward programming method for times 1 and 0 to find the optimal action functions, which will depend on the interval of values to which \(r\) belongs. (Hence you should be able to make statements of the form: "For state \(\mathbf{i}\), at time \(n\), action \(a\) is optimal for \(r\) belonging to a set; otherwise action \(b\) is optimal" for each of the two times and nine states.)

Exercises 13-17 lead you through another financial model. A risky asset moves in the manner of Figure 6.6. We follow the progress of an investor who owns a portfolio of some shares in this asset as well as a checking account \((0 \%\) interest) containing some amount of money at time 0 . The investor can decide to either invest in more shares of the same asset or to sell some number of the shares he holds currently. The investor is to decide how to change the portfolio and how to consume money as time progresses, in order to maximize the expected total consumption. In the finite time horizon investment problem, all wealth is consumed at the terminal time \(T\) and only then. (In a similar infinite horizon discounted problem, the investor maximizes the expected total discounted consumption over all periods.) To begin modeling this situation, let the information contained in the state of the process be given by

\(x=\) current price of risky asset;

\(r=\) current number of shares of risky asset;

\(y=\) current checking account balance.

Therefore, a state \(\mathbf{i}\) is a three-dimensional vector: \(\mathbf{i}=(x, r, y)\). The associated stochastic process has three component processes: \(X_{n}, R_{n}\), and \(Y_{n}\), indicating the share price and quantity held of the risky asset and the checking account balance at time \(n\). At a time such that the state of the process is \((x, r, y)\), the total wealth of the investor is \(r \cdot x+y\). We will suppose that as time progresses, the investor can shift money from the risky asset to the checking account and vice versa.