Implement an exploring reinforcement learning agent that uses direct utility estimation. Make two versions—one with a tabular representation and one using the function approxi-mator in Equation (22.9). Compare their performance in three environments:

a. The 4 × 3 world described in the chapter. 

b. A 10 × 10 world with no obstacles and a +1 reward at (10,10). 

c. A 10 × 10 world with no obstacles and a +1 reward at (5,5).