$($a$)$ In s$-$greedy action selection, tor the case of three actions and $s=0,3,$ what is the probability

that the greedy action is selected? Explain.

$($b$)$ Consider a $k-$armed band itproblem with $k=4$ actions, denoted $1,2,3,$ and $4,$ Consider

applying to this problem a bandit a logithm using s greedy action scection, sample$-$average

action$-$value estimates, and intial estimates of $Q1(a)=2,$ for all a$.$ Suppose the intial

sequence of actions and rewards is $A_1=1,R_1=2,A_2=2,R_2=2,A_1=2,R_1=1,A_1=2,R$

$=1,A_s=3,R=1,$ On some of these time steps the $s-$case may have occurred, causing an

action to be selected al random.

$($i$)$ On which time steps of this definitely occurWhy?

$($i$)$ On which lime steps could this possiby have ocourred? Why