2 MDPs +50 -1 - 1 -1 -1 -1 Start -50 +1 +1 +1 +1 +1 +1 (b) Figure 2: Figure 17.14(b) 1. Consider the 101 x 3 world shown in Figure 2. In the start state the agent has a choice of two deter- ministic actions, Up or Down, but in the other states the agent has one deterministic action, Right. Assuming a discounted reward function, for what values of the discount should the agent choose Up and for which Down? Compute the utility of each action as a function of 7 (Note that this simple example actually reflects many real-world situations in which one must weigh the value of an immediate action versus the potential continual long-term consequences, such as choosing to dump pollutants into a lake.)