t$,$ we will practice implementing the gradient descent algorithm for kernel SVM$.$

More specifically, you are asked to write either a Python class or a Python function, which takes:

$$ a set of feature vectors;

$$ the corresponding binary labels;

$$ the number of epochs;

$$ the learning rate parameter $\backslash$eta ;

$$ the regularization parameter $\backslash$lambda ; and

$$ the choice of the kernel, which can be either linear or polynomial

as the input, and output a set of dual variables $\backslash$alpha $=(\backslash$alpha $1,\backslash$alpha $2,...,\backslash$alpha m$)$ and a bias b based on the

gradient descent algorithm that we discussed in the lecture. In addition, your program should

also be able to make predictions for a given set of test feature vectors.

Use your program to complete the following two exercises on the Iris dataset. As usual, please

standardize the features before feeding them into the training algorithm.

Exercise $1(40$ Points$)$: Use your program to train a LINEAR$-$kernel SVM classifier that can be

used to classify between virginica and setosa. Starting from $0,$ try several choices of the regularization parameter $\backslash$lambda $.$ For each choice of $\backslash$lambda $,$ i$)$ adjust the learning rate parameter $\backslash$eta to ensure that

the gradient descent algorithm actually converges; and $2)$ draw the scatter plot of the training

dataset $($virginica and setosa only$)$ and plot the decision boundary in the same figure. Observe

how the choice of the regularization parameter $\backslash$lambda affects the output predictor and explain what

you have observed to the best of your knowledge.

Exercise $2(40$ Points$)$: Use your program to train a POLYNOMIAL$-$kernel SVM classifier that

can be used to classify between versicolor and the rest. Try several choices of the regularization

parameter $\backslash$lambda and the degree of the polynomial kernel k$.$ Pick one combination of $\backslash$lambda and k that

you feel performs the best. Draw the scatter plot of the training dataset $($versicolor and rest$)$

and plot the decision boundary in the same figure. Compare the performance of your kernel

SVM predictor with the Adaline that you trained from the previous assignment in terms of the

training error rate.