Question: Consider the strongly convex function y[x] = (x 2). 1. Analytically obtain the global minimizer of the function via taking the gradient of the

Consider the strongly convex function y[x] = (x 2). 1. Analytically obtain 

Consider the strongly convex function y[x] = (x 2). 1. Analytically obtain the global minimizer of the function via taking the gradient of the function and finding the root of the gradient. 2. Assume that you have written a code for gradient descent over this function and xo=0 is chosen as the initial point. Mimic what the code would do and fill out the following steps for three different choices of step size (n = {0.001, 0.1,5}): Iteration 1: x = xo nVy[xo] X =? Iteration 2: x = x nVy[x] x =? Iteration 2: x3 = x nVy[x] X3 =? Iteration 2: x4 = x3 nVy[x3] X4 =? Iteration 2: *5 = x4 nVy[x4] X5 =? you need to generate 15 numbers (5 iterations for each choice of step size) " 3. What is the difference in terms of the behavior of the solutions (i.e., x, x5) for the three aforementioned choices of step size? Hint: How is the sequence approaching or bouncing back and forth around the global minimizer? Hint: 4. Given the above intuition, what do you think about this statement: "It is always better to choose a smaller step size for gradient descent". Hint: think about the speed of conver- gence vs. accuracy! Sometimes there is no definite Yes/No answer and we have to study the tradeoff.

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Computer Network Questions!