Figure 17.13 shows two MDPs: one, M, represents a two-armed bandit where one has the choice to continue with the first arm or to switch permanently to a second arm with fixed reward λ; the other, a restart MDP M^s , gives one a choice to continue with the first arm or restart the sequence. The figure illustrates the construction of M^s for the case where M has a deterministic reward sequence and just two arms (including the λ-arm). Explain how to construct M_s when M has k + 1 arms (including the λ-arm) and each arm is a general MRP. Show that the value of M^s equals the minimum value of λ such that one would be indifferent in M between pulling the best arm and switching to the λ-arm forever.