For all problems, you can use any software to code the dynamic programming equations and solve. However, you cannot use an optimization solver to solve the problems. Please, provide printouts for your solution. Question 1. Consider the optimization problem max 4z1 + 5z2 + 4z3 + 874 + 675 421 +672 + 5z3 +974 + 775 0, zj e Z j = 1,...,5. (1) Assume that all Z; may take only nonnegative integer values. (Q1.a) Formulate a dynamic programming problem for solving problem (1). Clearly describe the state space, the control space, the feasible action mapping, and the dynamic programming equations. (Q1.b) Use the dynamic programming equations to obtain the optimal solution. For all problems, you can use any software to code the dynamic programming equations and solve. However, you cannot use an optimization solver to solve the problems. Please, provide printouts for your solution. Question 1. Consider the optimization problem max 4z1 + 5z2 + 4z3 + 874 + 675 421 +672 + 5z3 +974 + 775 0, zj e Z j = 1,...,5. (1) Assume that all Z; may take only nonnegative integer values. (Q1.a) Formulate a dynamic programming problem for solving problem (1). Clearly describe the state space, the control space, the feasible action mapping, and the dynamic programming equations. (Q1.b) Use the dynamic programming equations to obtain the optimal solution