Question: 2. For a > 0, a smooth function f : R )- R is a-conoez'c if for all 3:, y E R, M) 2 ms)

 2. For a > 0, a smooth function f : R\"

)- R is a-conoez'c if for all 3:, y E R", M)

2. For a > 0, a smooth function f : R\" )- R is a-conoez'c if for all 3:, y E R", M) 2 ms) + + Elly 3H5. In this exercise we will show a better bound on convergence rate of gradient descent (GD) for such functions. Let f : R\" ) R be a cr-convex iii-smooth function. Let 33'\" be an optimal minimizer for j. Consider the GD algorithm with starting point In. and step-size t = 1/3. Show that after it iterations we have 59 05 Hits: :r:''||% g \"In M3 (1 3) - In other words, the distance of the k'th iterate to the optimal decreases exponentially in the number of iterations. [4 points] [Hint: You may use the fact that Vf(3:"') = 0.] Remark: Note that the number of iterations to get error a for such functions is O(log(1/s)) for fixed 339,0, .3; this is exponentially better than the bound for general covex functions which was 0(1/5)

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!