Exerciae $3($Data Separation$),$ Assume we have $n$ data points $x_{i}in{R}^d,$ with label $y_{i}in\{-1,1\}.$ We ane searching for an mypen$-$plane defined by iva normal $,$ which separates the points according to their label. Ideally, we would like to have

${}^2{x}_i-1=>y_i=-1$ and ${}^T{x}_{i}1=>y_i=1.$

Unfortunately, this condition is rarely met with real$-$life problems. Instead, we solve an optimization problem which minimizes the gap between the hyper$-$plane and the miss$-$classified points. To do so$,$ we will use a specific loss fumetion

$C({}_i,x_i,y_i)=$max$\{0$;$1-y_i({}^T{x}_i)\}$

$1$

which is equal to $0$ when the point $x_{\text{;}}$ is well$-$classified $($the sign of ${}^T{x}_{\text{;}},$ and $y_i$ is the same$),$ but is strictly positive when the sign of ${}^T{x}_i$ and $y_i$ is different. To improve the performances, instead of minimizing the los function alone, we alho use a quadratic regularizer as follow.

$mi{n}_{-}\frac{1}{n}{\displaystyle \underset{k=1}{\overset{n}{}}}C(,x_i,y_i)+\frac{}{2}|||{|}_2^2$

where to is the regularization parameter.

Consider the following quadratic optimization problem $(1$ is a vector full of ones$).$

min$\frac{1}{n}{1}^{T_2}+\frac{1}{2}||{}_i|{|}_2^2$

$n_{i}1-m_i({}^{r_i}{n}_i),$AAi$=1$dotsn

$20$

Esplain why problem $($Sep$.2)$ solves problem $($Sep$.1).$

$2.$ Compute the duat of $($Sep$.2),$ and try to reduce the number of variables. Use the notations ${}_i$ and $-$ tor the dual variables.