Consider an error function (W - 3) (w -4) (w-3) (w - 4) 4 + 9...

Transcribed Image Text:

Consider an error function (W₁ - 3)² (w₂ -4)² (w₁-3) (w₂ - 4) 4 + 9 6 . Different variants of gradient descent algorithm can be used to minimize this error function w.r.t (w₁, W2). Assume (w₁, W₂) = (1, 1) at time (t-1) and after update (w₁, W2) = (1.5, 2.0) at time (t). Assume a = 1.5, B =0.6, n = 0.3. 1. E(W₁, W₂) = 0.05 + Compute the value that minimizes (w1 , w2). Compute the minimum possible value of error. 2. What will be value of (w1, w2 ) at time (t + 1) if standard gradient descent is used? 3. What will be value of (w1, w2 ) at time (t + 1) if momentum is used? 4. What will be value of (w1, w2 ) at time (t + 1) if RMSPRop is used? 5. What will be value of (w1, w2 ) at time (t + 1) if Adam is used? Consider an error function (W₁ - 3)² (w₂ -4)² (w₁-3) (w₂ - 4) 4 + 9 6 . Different variants of gradient descent algorithm can be used to minimize this error function w.r.t (w₁, W2). Assume (w₁, W₂) = (1, 1) at time (t-1) and after update (w₁, W2) = (1.5, 2.0) at time (t). Assume a = 1.5, B =0.6, n = 0.3. 1. E(W₁, W₂) = 0.05 + Compute the value that minimizes (w1 , w2). Compute the minimum possible value of error. 2. What will be value of (w1, w2 ) at time (t + 1) if standard gradient descent is used? 3. What will be value of (w1, w2 ) at time (t + 1) if momentum is used? 4. What will be value of (w1, w2 ) at time (t + 1) if RMSPRop is used? 5. What will be value of (w1, w2 ) at time (t + 1) if Adam is used?

Answer rating: 100% (QA)

To find the minimum set the partial derivatives of E w w with respect to w and w to zero and s... View the full answer