Question: Implement gradient descent for the least squares problem in any programming language you wish, and run it on our synthetic test data. Write a function

Implement gradient descent for the least squares problem in any programming language you wish, and run it on our synthetic test data. Write a function with a signature similar to gradient_descent(X, y, step_size, precision) so that can be run on more than one data set. Note: the least squares problem here is .
Try a number of different choices of step size. You are looking for the optimal step size, which is the choice of step size that results in the fewest number of iterations before convergence. You should use a termination condition that uses a relative precision of or better for , and can use an initial starting of all zeros (although, in practice, the result depends only very weakly on these choices).
What is the optimal step size that you find? (Please enter the value with a precision of at least one significant figure, your answer will be graded with a 25% tolerance.)

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!