Problem 3. We have m identical items which we want to sell over the next N days. At each day we may sell at most one item, and the probability of sale is equal to e-u, where u is the price set for this day. Our objective is to find the optimal policy of setting prices at each day, so as to maximize the expected revenue over N days. We assume that N > m. (a) Formulate the corresponding Markov decision problem. Clearly define the state space, control space, transition probabilities, and the reward function. (b) Solve the problem numerically for the following data: m = 5, N = 10. (c) (Bonus Problem. For t = 1,2, ... , N and all x > 0, establish the equations: uf(x) = 1 + v*+1(x) v*+1(x - 1), vit = v1+1(x) +e-ui (2) , (1) where v*(x) is the optimal value function for state x at time t and u(x) is the optimal control for state x at time t. Problem 3. We have m identical items which we want to sell over the next N days. At each day we may sell at most one item, and the probability of sale is equal to e-u, where u is the price set for this day. Our objective is to find the optimal policy of setting prices at each day, so as to maximize the expected revenue over N days. We assume that N > m. (a) Formulate the corresponding Markov decision problem. Clearly define the state space, control space, transition probabilities, and the reward function. (b) Solve the problem numerically for the following data: m = 5, N = 10. (c) (Bonus Problem. For t = 1,2, ... , N and all x > 0, establish the equations: uf(x) = 1 + v*+1(x) v*+1(x - 1), vit = v1+1(x) +e-ui (2) , (1) where v*(x) is the optimal value function for state x at time t and u(x) is the optimal control for state x at time t