Question $2$: Support Vector Machine and Decision Trees

$[$overall $33$ marks$]$

a$.$ Draw a two dimensional plot, in which the horizontal axis is $x_1$ indicating the first feature

value and the vertical axis is $x_2$ indicating the second feature value. Then add $20$ data

points that are sampled from two classes: $10$ data points in each of the positive $($annotate

as circles in the plot$)$ and negative $($annotate as crosses in the plot$)$ classes. These $20$ data

points have to be plotted so that they are not linearly separable, and ensure that there are

at least two misclassified data points using a linear classifier.

$[3$ Marks$]$

b$.$ Copy the plot you produced in $($a$),$ draw the decision boundary and margin with a brief

explanation to illustrate how soft margin SVM works to classify the two classes. Explain

the effects of any hyper$-$parameters used in the soft margin SVM algorithm.

$[7$ Marks$]$

c$.$ Explain what kernel method is in the context of SVM algorithm, and how the kernel trick

can help to achieve efficient computation.