Consider two coins, and write the random variable for the payoff from each

coin as $x^{(1)}$ and $x^{(2)}.$ The ground truth distribution for each coin is $P(x^{(1)}=0)=0\mathrm{.}2,P(x^{(1)}=1)=0\mathrm{.}8,$

$P(x^{(2)}=0)=0\mathrm{.}6$ with $P(x^{(2)}=1)=0\mathrm{.}4.$ Plot the total reward of $T$ rounds of play

$J(T)={\displaystyle \underset{i=1}{\overset{T}{}}}{x}^{(c_i)}$

where $c_{i}in\{1,2\}$ is the coin choice, as the number of plays $T$ goes from $10$ to $1000$ for each of the following strategies.

Keep in mind that $J(T)$ is a random variable. The $x-$axis should be $T$ and $y-$axis is $J(T),$ and plot everything on the

same graph so that the curves can be compared.

Explore$-$then$-$commit with is the ceiling function that gives you an integer$).$

Explore$-$then$-$commit with $N=|$~$\frac{1}{2}{T}^{\frac{2}{3}}(logT{)}^{\frac{1}{3}}$~$|,$ where $log$ is the natural logarithm.

The $-$Greedy strategy with $=0\mathrm{.}2.($With $1-$ probability play the currently$-$best coin, and with $$ probability

the other$).$

The Upper Confidence Bound strategy, which plays the coin iin$\{1,2\}$ that maximizes ${\stackrel{}{x}}^{(i)}+\sqrt[\phantom{2}]{\frac{2logT}{N^{(i)}}}$ in each

round $T.$

Design some new plots to show the regret of these strategies and explain.