In this problem set you will get some practice with gradient descent and sub-gradient descent. There is a template for setting up your code and plotting your results at the end of this notebook, which you may find useful. Note that it is only a suggestion -- you are not required to use it.

Consider three quadratics, as given by:

f_i (x) = 1/2 x^T Q_i x + q^Ti + c_i

i = 1,2,3

import numpy as np Q1 = np.array([[ 1.17091573, -0.03686123, -0.1001259 ], [-0.03686123, 1.03835691, 0.17285956], [-0.1001259 , 0.17285956, 1.06072736]]) Q2 = np.array([[ 15.27095759, -1.97915834, -14.22190153], [ -1.97915834, 0.34660759, 1.91586927], [-14.22190153, 1.91586927, 15.76943482]]) Q3 = np.array([[28.59657006, 0.3684004 , 0.90750259], [ 0.3684004 , 28.11480924, 0.81866989], [ 0.90750259, 0.81866989, 28.7886207 ]]) q1 = np.array([-4.68686663, -0.89027884, -1.57340281]) q2 = np.array([ 6.75973216, 1.23223936, -0.87956749]) q3 = np.array([ 0.8684369 , -4.69970837, -1.09690726]) c1 = 1.61888566; c2 = -2.66426693; c3 = 0.84184738;

Part (A)

Implement gradient descent with a step-size of 0.1 for all three quadratics, and plot function value versus iteration number. Explain what you see.

Part (B)

Find a step-size for each quadratic (it can be different for each), and implement gradient descent with this stepsize. Plot function value versus iteration number. Try to pick a step size that gives you fast convergence. How did you pick it?

Part (C)

Even with your carefully tuned choice of step size, you will see that the rates of convergence are not the same, explain why this is the case.

In this problem set you will get some practice with gradient descent and sub-gradient descent. There is a template for setting up your code and plotting your results at the end of this notebook, which you may find useful. Note that it is only a suggestion -- you are not required to use it. Problem 1: Gradient Descent Consider three quadratics, as given by: fi(x)=21xQix+qix+ci, for the three triples (Qi,qi,ci),i=1,2,3 given below. Implement gradient descent with a step-size of 0.1 for all three quadratics, and plot function value versus iteration number. Explain what you see. Part (B) Find a step-size for each quadratic (it can be different for each), and implement gradient descent with this stepsize. Plot function value versus iteration number. Try to pick a step size that gives you fast convergence. How did you pick it? Part (C) Even with your carefully tuned choice of step size, you will see that the rates of convergence are not the same, Explain why this is the case