$($b$)10pts$ Let ${}^k$ be the policy at the end of $k-$th iteration. Show that

$||V^{{}_k}-V^{***}||\frac{2{}^k}{1-}||V_1-V_0||$

Hint: use the contracion property of Bellman operators. The above is Theorem $6\mathrm{.}3\mathrm{.}3,$ Section $6\mathrm{.}3\mathrm{.}2$

in Puterman's book $($Markov Decision Processes: Discrete Stochastic Dynamic Programming. John

Wiley $83$ Sons, Inc., New York, NY$,$ USA, $1$st edition, $1994).$ Try to prove it yourself. If you need

help, you can read the proof from the book.