$($a$)(10$ points$)$ Formulate the state space, action space and rewards for each state and

action pair for this dynamic decision$-$making problem.

$($b$)(10$ points$)$ Visualize the transition probabilities in a graph with the nodes being the

states and the arcs stating the action and transition probabilities. You can use the

template provided in the appendix. Note the additional $$Harvested $($H$)$ state, which

is a terminal $($sink$)$ state with no allowed action and no immediate reward.

$($c$)(10$ points$)$ Write down the Bellman optimality equation for each state explicitly and

determine the optimal actions for each state. What is the optimal harvest time point

for this batch fermentation?