Answer the question step by step and with a clear calculation and explanation.

Question 2. Stochastic Dynamic Programming A gambler has $4, and he can bet up to 4 times with the goal of maximizing the probability of ending with exactly $8. In each bet, the probability of winning is 0.6 and the probability of losing is 0.4 . If the gambler wins the bet, he gets twice the value of the bet back, but if he loses, he gets nothing back (including the bet). In addition, the gambler can only bet value up to what he has available at the beginning of each betting time. a. Identify stage, state, and decision of the problem. b. Let ft(s) be the maximal probability that, by the end of stage 4 , the gambler has $8, given that he has $s at the beginning of stage t. Let bt(s) be the set of all possible bet value at stage t, given that he has $s at the beginning of that stage. Write out bt(s) in terms of s and identify the recursion of ft(s) c. Compute the final stage probability which can be denote as f5(s) for s={0,1,2,,8}. (Hint: beginning of stage 5 is equivalent to end of stage 4 ) d. Compute the maximal probability that the gambler has $8 by the end of stage 4 given that he has $ at the beginning of stage 4 , which is f4(s)s={0,1,,8}, using the recursion in (b)