Problem 1 [50 points]: Consider the following optimization problem where mi nf x,y z ) x...

Transcribed Image Text:

Problem 1 [50 points]: Consider the following optimization problem where mi nf x,y z ) x y z f (x, y, z) = (x-2)+ 10(y-3) + (z - 1) It is obvious that the solution to the above optimization problem is (xyz) = (2,3,1). The goal of this exercise is to demonstrate that gradient descent (GD) converges to the above solution. a) [written, 5 pts] Let a = 0.01, with initial guess of (x oy 0, 0) = (1,2,2), find (x1,y 1, z 1), (x 2y 2, z 2), and (x 3,y 3, z 3). What is the value of the function f (x,y,z) at (x 0, y 0, 0),(x1,y 1, 1), (x 2 y 2,2 2), and (x 3, 3, z 3)? Is GD improving the candidate solution after each iteration? b) [written, 5 pts] Repeat a) with a = 0.025 c) [written, 5 pts] Repeat a) with a = 0.1 d) [written, 5 pts] From parts a)-c), what is the effect of changing the learning rate a? e) [programming, 20 pts] For the same initial guess in part a), for each of the above learning rates, implement the GD to find the optimal solution of the above optimization problem with error criterion If (x n, y n, z n ) - f (xn1, y n-1,2 n-1) < 108. For each a, what is the required number of iterations ? If the algorithm diverges, you may stop and state that it does not converge. f) [written, 10 pts] Analytically, verify that Vf (2,3,1) = [000] Problem 1 [50 points]: Consider the following optimization problem where mi nf x,y z ) x y z f (x, y, z) = (x-2)+ 10(y-3) + (z - 1) It is obvious that the solution to the above optimization problem is (xyz) = (2,3,1). The goal of this exercise is to demonstrate that gradient descent (GD) converges to the above solution. a) [written, 5 pts] Let a = 0.01, with initial guess of (x oy 0, 0) = (1,2,2), find (x1,y 1, z 1), (x 2y 2, z 2), and (x 3,y 3, z 3). What is the value of the function f (x,y,z) at (x 0, y 0, 0),(x1,y 1, 1), (x 2 y 2,2 2), and (x 3, 3, z 3)? Is GD improving the candidate solution after each iteration? b) [written, 5 pts] Repeat a) with a = 0.025 c) [written, 5 pts] Repeat a) with a = 0.1 d) [written, 5 pts] From parts a)-c), what is the effect of changing the learning rate a? e) [programming, 20 pts] For the same initial guess in part a), for each of the above learning rates, implement the GD to find the optimal solution of the above optimization problem with error criterion If (x n, y n, z n ) - f (xn1, y n-1,2 n-1) < 108. For each a, what is the required number of iterations ? If the algorithm diverges, you may stop and state that it does not converge. f) [written, 10 pts] Analytically, verify that Vf (2,3,1) = [000]