Consider the shown $(3$x$3)$ game world that has $9$ states $\{$A$,$ B$,$ C$,$ D$,$ E$,$ F$,$ G$,$ H$,$ I$\}$ and four actions $($right$,$ left,up$,$ down$).$ In every new episode, the game starts by choosing a random state and ends when state F isreached, for which the player receives a reward of $+10.$ For all other actions that do not lead to state F$,$ thereward is $-1.$ Shown below, Q$,$ is the Q function after initial training using the Q$-$learning algorithm.