Implement an exploring reinforcement learning agent that uses direct utility estimation.

Make two versions-ne with a tabular representation and one using the function approximator in Equation (21.9). Compare their performance in three environments:

a. The 4 x 3 world described in the chapter.

b. A 10 x 10 world with no obstacles and a +1 reward at (10,lO).

c. A 10 x 10 world with no obstacles and a +1 reward at (5,5).