In this assignment, you will explore two cornerstone algorithms of machine learning: K$-$Nearest Neighbors $($KNN$)$ and Support Vector Machines $($SVM$).$

Through hands$-$on tasks, you will implement these algorithms to tackle classification problems, comparing their effectiveness and learning to fine$-$tune their parameters. This will not only enhance your understanding of the theoretical underpinnings of KNN and SVM but also provide practical experience in applying these techniques to real$-$world datasets.

Assignment Requirements:

The students are required to complete the functions provided in this notebook. It is crucial that you do not change the function definition $($the names of the function and their paramters$).$ Each function should return exactly what is mentioned in its description, and the outputs must be in the same order as specified.

Submission Instructions:

You are required to submit your work in the form of a Python file. To convert your Jupyter notebook into a Python script, follow the steps appropriate for your working environment:

Important Note: This assignment will be graded automatically by executing the functions defined in your submission. Therefore, it is imperative that the function names remain unchanged and that your functions return exactly what is requested, in the exact order it is requested.

Failure to adhere to these instructions may result in points being deducted from your assignment. Please ensure that your submission meets these requirements to avoid any penalties.

$1\mathrm{.}3$ Using the KNN Classifier for Classification

The K$-$Nearest Neighbors $($KNN$)$ classifier offers a straightforward approach to classification tasks by leveraging the properties of nearby data points. This task involves using the KNN classifier to predict the class of new instances based on the $\text{'}K\text{'}$ nearest neighbors and evaluating its performance on a test set.

Objective:

Use the KNN classifier to make predictions on a test dataset.

Evaluate the classifier's performance using metrics such as accuracy, precision, recall, true positives, and true negatives.

Requirements:

Implement a function named evaluate$\_$knn$\_$classifier.

Parameters:

x$\_$train : Training data features as a numpy array.

y$\_$train : Training data labels as a numpy array.

x$\_$test : Test data features as a numpy array.

y$\_$test : Test data labels as a numpy array.

best$\_$k: The optimal number of neighbors as an integer. This should be taken from the output of last function.

Return:

The function should return the evaluation metrics for the test set: accuracy, precision, recall, true positives, and true negatives.

$2\mathrm{.}1$ Utilizing the SVM Classifier for Classification with a Linear Kernel

You will focus on using an SVM with a linear kernel to classify data. The objective is to train the SVM classifier using the training dataset and evaluate its performance on a test dataset.

Objective:

Train an SVM classifier with a linear kernel on the training dataset.

Evaluate the classifier's performance on the test dataset.

Return the slope and intercept of the decision boundary, along with key performance metrics: accuracy, precision, recall, false positives, and false negatives.

Requirements:

Implement a function named evaluate$\_$svm$\_$classifier $.$

Parameters:

x$\_$train : Training data features as a numpy array.

y$\_$train : Training data labels as a numpy array.

x$\_$test : Test data features as a numpy array.

y$\_$test : Test data labels as a numpy array.

SVM Kernel should be linear

Return:

Slope and intercept of the decision boundary.

Accuracy, precision, recall of the classifier on the test set.

Number of false positives and false negatives.