Consider a stochastic world with two states and two actions. The agent performs actions

and observes rewards and transitions $-$ see below.

a$.$ At each step, current state $(S_i),$ reward $(R=r),$ action, and resulting state

$(a_k$:$S_i{S}_j)$ are provided. Perform Q$-$learning using a learning rate of $=0\mathrm{.}5$

and a discount factor of $=0\mathrm{.}5$ for each step. The Q$-$table entries are initialized

to zero. Note that the following actions are performed in a row.

b$.$ What is the optimal policy after the above actions?