Question: solve part B only 2. In the gradient descent (2), for distance between a and a(*), we used the 2-norm, which can be replaced by

solve part B only

solve part B only 2. In the gradient descent (2), for distance

2. In the gradient descent (2), for distance between a and a(*), we used the 2-norm, which can be replaced by other norms to obtain variants of the gradient descent algorithm. (a) Prove that (2) is equivalent to (k+1) = arg min (Vf(a(*) ), x - x )), subject to x - 2 2 5axlVf(z(*))|2. (3) IER" ( Hint: Use Cauchy-Schwartz inequality.) (b) We may replace the 2-norm in (3) by 1-norm, i.e., x(*+1) = arg min (Vf(a(*) ), x - 2*)), subject to |x - x)|1 5 axlVf(a(*))|x. (4) ER" Give an explicit expression of a(*+1) in (4). (Hint: Use the inequality (x, y) | - Ixlloollyll with equality attained if x = c . sign(y) or y = C . eimax(2) for some c E R. Here sign(.) takes the sign of each entry of a vector, and imax(2) = arg max, Ijl.)

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!