1 An exponential loss function f(w) is defined as f(w) e 2(w 1), w 1 ew 1, w1 a) Is f(w) convex Why Hint Graph the function b) Is f(w) differentiable everywhere If not, where not c) The differential set of (w) is the set of subgradients v af (w) for which f(u) f(w) (u w)Tv Find the differential set for f(w) as a function of w 2 We are trying to predict whether a certain chemical reaction will take place as a function of our experimental conditions temperature, pressure, concentration of catalyst, and several other factors For each experiment i 1, , m we record the experimental conditions in the vector x R and the outcome in the scalar b 1,1 ( 1 if the reaction occurred and 1 if it did not) We will train our linear classifier to minimize hinge loss Namely, we solve m minimize (1 bxw) w i 1 where (u) max(0, u) is the hinge loss operator a) Derive a gradient descent method for solving this problem Explicitly give the computations required at each step Note you may ignore points where the function is non differentiable b) Explain what happens to the algorithm if you land at a wk that classifies all the points perfectly, and by a substantial margin 3 You have four training samples y 1, x1 , Y2 2, X2 , Y3 2 2 1, x3 and y4 2,x4 0 Use cyclic stochastic gradient descent to find the first two updates for the LASSO problem miny Xw 2 W 1 assuming a step size of T six updates 1 and w(0) 0 Also indicate the data used for the first

Question

1  An exponential loss function f(w) is defined as f(w) e 2(w 1), w  1   ew 1, w1 a) Is f(w) convex  Why  Hint  Graph the function  b) Is f(w) differentiable everywhere  If not, where not  c) The  differential set  of (w) is the set of subgradients v af (w) for which f(u) f(w)   (u   w)Tv  Find the differential set for f(w) as a function of w  2  We are trying to predict whether a certain chemical reaction will take place as a function of our experimental conditions  temperature, pressure, concentration of catalyst, and several other factors  For each experiment i   1,   , m we record the experimental conditions in the vector x  R  and the outcome in the scalar b   1,1  ( 1 if the reaction occurred and  1 if it did not)  We will train our linear classifier to minimize hinge loss  Namely, we solve  m minimize (1 bxw)  w i 1 where (u)   max(0, u) is the hinge loss operator a) Derive a gradient descent method for solving this problem  Explicitly give the computations required at each step  Note  you may ignore points where the function is non differentiable  b) Explain what happens to the algorithm if you land at a wk that classifies all the points perfectly, and by a substantial margin  3  You have four training samples y   1, x1   , Y2   2, X2   , Y3  2  2  1, x3   and y4    2,x4   0 Use cyclic stochastic gradient descent to find the first two updates for the LASSO problem miny Xw  2  W  1 assuming a step size of T six updates    1 and w(0)   0  Also indicate the data used for the first

SolutionInn · Accepted Answer

The Answer is in the image, click to view ...

Question: 1. An exponential loss function f(w) is defined as f(w) e-2(w-1), w <1 { ew-1, w1 a) Is f(w) convex? Why? Hint: Graph the

Step by Step Solution

Students Have Also Explored These Related Mechanical Engineering Questions!