$($a$)$ Define briefly what is Dimensionality Reduction $($DR$).$ Discuss the role of Pearson

correlation coefficient in dimensionality reduction and compare it with another

unsupervised machine learning method to achieve the same goal.

$(5$ marks$)$

$($b$)$ Discuss feature reduction and its key steps in LASSO.

$($c$)$ Explain Kernel Trick in Support Vector Machines and its importance to computation

reduction. Does it apply to other Machine Learning algorithms?

$(5$ marks$)$

$($d$)$ The three training vectors are:

$x_1={\left[\begin{array}{cc}0\mathrm{.}0& 0\mathrm{.}0\end{array}\right]}^T,y_1=-1,$

$x_2={\left[\begin{array}{cc}-1\mathrm{.}0& 0\mathrm{.}0\end{array}\right]}^T,y_2=+1,$

$x_3={\left[\begin{array}{cc}0\mathrm{.}0& 1\mathrm{.}0\end{array}\right]}^T,y_3=+1,$

The objective function is:

$L()={\displaystyle \underset{1}{\overset{3}{}}}{}_i-\frac{1}{2}{\displaystyle \underset{i=1}{\overset{3}{}}}{\displaystyle \underset{j=1}{\overset{3}{}}}{}_i{}_j{y}_i{y}_j(x_i^T{x}_j)$

Subject to ${}_{i}0$ and ${\displaystyle \underset{i=1}{\overset{3}{}}}{}_i{y}_i=0.$

Assuming equality constraints, i$.$e$.,{\displaystyle \underset{i=1}{\overset{3}{}}}{}_i{y}_i=0,$ a Lagrange multiplier, say $"$b$"$ can

be introduced to derive the new Lagrangian function.

$($i$)$ State the new Lagrangian function

$($ii$)$ Derive the decision boundary