We have argued in class that a deterministic policy may result in extremely suboptimal outcomes.

This is especially true when the state transition model is adversarial, i$.$e$.$ the next state is chosen by an

adversary who wants to minimize your reward. Consider the game of scissors$/$paper$/$stone played repeatedly

$($infinitely many times$).$ At each turn, Player $1$ and Player $2$ pick either $$scissors$$paper$$ or $$stone$.$ The

states and rewards are given as hstatei $->$ reward below