3. (10 points) In this question you will see that soft-SVM can be formulated as solving...

Transcribed Image Text:

3. (10 points) In this question you will see that soft-SVM can be formulated as solving a regularized ERM rule, which uses a popular error function called hinge loss (see https: //en.wikipedia.org/wiki/Hinge_loss). Consider the following optimization problem: min (A||||2 + w,b, 1 N i) N i= s.t. y(x+b) 1 i and i > 0 for all i, where is a parameter, as opposed to 1 N (1) min A|||| + Chinge ((w, b), (i, Yi)) w,b i=1 where Chinge ((w, b), (x, y)) = max {0, 1 y(wx+b)} (2) Prove that Problem (1) and Problem (2) are equivalent, and explain why the latter is in fact a regularized ERM rule. 3. (10 points) In this question you will see that soft-SVM can be formulated as solving a regularized ERM rule, which uses a popular error function called hinge loss (see https: //en.wikipedia.org/wiki/Hinge_loss). Consider the following optimization problem: min (A||||2 + w,b, 1 N i) N i= s.t. y(x+b) 1 i and i > 0 for all i, where is a parameter, as opposed to 1 N (1) min A|||| + Chinge ((w, b), (i, Yi)) w,b i=1 where Chinge ((w, b), (x, y)) = max {0, 1 y(wx+b)} (2) Prove that Problem (1) and Problem (2) are equivalent, and explain why the latter is in fact a regularized ERM rule.