In this programming task you will gain familiarity with k-nearest neighbor classification, the sklearn machine learning library, and working with a handwriting recognition dataset.

For this program, compare the accuracy of four classifiers for correctly classifying the hand-written number from the digits dataset available as a sklearn library (see https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html for details). The classifiers are:

• Decision tree (you can use the sklearn library for this)
• K nearest neighbors with 5 neighbors (you can use the sklearn library for this)
• Majority classifier (you can use the sklearn library for this)
• Your own implementation of a KNN classifier (do not use the sklearn library for this). This classifier should compute Euclidean distance between pairs of points and take the number of neighbors to consider as a parameter.

To report performance, randomly select 2/3 of the data points to use for training and 1/3 to use for testing. Repeat 3 times and report accuracy results averaged over the 3 trials. Compare accuracy results for the classifiers. For your KNN implementation, try different values for k including 1, 3, 5, 7, and 9. Argue which value of k you would choose and why.

import matplotlib.pyplot as plt

from matplotlib import lines

from mpl_toolkits.mplot3d import Axes3D

# you need to define the min and max values from the data

step_size = 0.05

xx, yy, zz = np.meshgrid(np.arange(x_min, x_max, step_size),

np.arange(y_min, y_max, step_size),

np.arange(z_min, z_max, step_size))

# the colors of the plot (parameter c)

# should represent the predicted class value

# we found this linewidth to work well

fig = plt.figure()

ax = fig.add_subplot(111, projection='3d')

ax.scatter(xx, yy, zz, c=c_pred, marker='s', edgecolors='k', linewidth=0.2)

# you will want to enhance the plot with a legend and axes titles

plt.show()