1. An exponential loss function f(w) is defined as f(w) e-2(w-1), w ...

Transcribed Image Text:

1. An exponential loss function f(w) is defined as f(w) e-2(w-1), w <1 { ew-1, w1 a) Is f(w) convex? Why? Hint: Graph the function. b) Is f(w) differentiable everywhere? If not, where not? c) The "differential set" of (w) is the set of subgradients v af (w) for which f(u) f(w) + (u - w)Tv. Find the differential set for f(w) as a function of w. 2. We are trying to predict whether a certain chemical reaction will take place as a function of our experimental conditions: temperature, pressure, concentration of catalyst, and several other factors. For each experiment i = 1,..., m we record the experimental conditions in the vector x; R" and the outcome in the scalar b; {1,1} (+1 if the reaction occurred and -1 if it did not). We will train our linear classifier to minimize hinge loss. Namely, we solve: m minimize (1-bxw)+ w i=1 where (u)+ =max(0, u) is the hinge loss operator a) Derive a gradient descent method for solving this problem. Explicitly give the computations required at each step. Note: you may ignore points where the function is non-differentiable. b) Explain what happens to the algorithm if you land at a wk that classifies all the points perfectly, and by a substantial margin. 3. You have four training samples y = 1, x1 = , Y2 = 2, X2 = , Y3 -2 -2 -1, x3 = and y4 = -2,x4 = 0 Use cyclic stochastic gradient descent to find the first two updates for the LASSO problem miny Xw|+2||W||1 assuming a step size of T six updates. = 1 and w(0) = 0. Also indicate the data used for the first 1. An exponential loss function f(w) is defined as f(w) e-2(w-1), w <1 { ew-1, w1 a) Is f(w) convex? Why? Hint: Graph the function. b) Is f(w) differentiable everywhere? If not, where not? c) The "differential set" of (w) is the set of subgradients v af (w) for which f(u) f(w) + (u - w)Tv. Find the differential set for f(w) as a function of w. 2. We are trying to predict whether a certain chemical reaction will take place as a function of our experimental conditions: temperature, pressure, concentration of catalyst, and several other factors. For each experiment i = 1,..., m we record the experimental conditions in the vector x; R" and the outcome in the scalar b; {1,1} (+1 if the reaction occurred and -1 if it did not). We will train our linear classifier to minimize hinge loss. Namely, we solve: m minimize (1-bxw)+ w i=1 where (u)+ =max(0, u) is the hinge loss operator a) Derive a gradient descent method for solving this problem. Explicitly give the computations required at each step. Note: you may ignore points where the function is non-differentiable. b) Explain what happens to the algorithm if you land at a wk that classifies all the points perfectly, and by a substantial margin. 3. You have four training samples y = 1, x1 = , Y2 = 2, X2 = , Y3 -2 -2 -1, x3 = and y4 = -2,x4 = 0 Use cyclic stochastic gradient descent to find the first two updates for the LASSO problem miny Xw|+2||W||1 assuming a step size of T six updates. = 1 and w(0) = 0. Also indicate the data used for the first