Consider the MDP shown.

$($a$)$ What are the various deterministic policies possible in this MDP$?$

$($b$)$ What is the optimal average reward in this MDP$?$

$($c$)$ Which of the policies are gain optimal?

$($d$)$ Compute the average adjusted value function under the bias optimal policy.

$($e$)$ For what values of gamma are each of the policies optimal under a discounted reward formu$-$

lation?