Objectives:

Apply and evaluate three different machine learning methods on image recognition dataset.

Compare the effectiveness of these methods using various performance metrics.

Understand the challenges involved in applying machine learning to digit recognition.

Dataset: Each line consists of the digit id $(0-9)$ followed by the $256$ grayscale values. There are $7291$ training observations and $2007$ test observations, distributed as follows:

$0123456789$ : Total

Train: $11941005731658652556664645542644$ : $7291$

Test: $359264198166200160170147166177$ : $2007$

The test set is notoriously "difficult", and a $2\mathrm{.}5\%$ error rate is excellent. These data were kindly made available by the neural network group at AT&T research labs.

Instructions

A$.$ Feature Selection: Utilize any appropriate feature selection method to select relevant features before training the model.

B$.$ Machine Learning Models Implementation:

Implement any three machine learning models we covered in our classes like: KNN$,$ANN, SVM$,$ LDA, QDA, Linear Regression, etc. You may also try implementing other methods such as XGBoost, Random Forest, etc.

You may use Rapid Miner, WEKA, Google CoLab, Python Notebook, Python libraries such as Scikit$-$learn for implementing these models.

C$.$ Model Evaluation:

Evaluate each model using $5-$fold Cross Validation on the following performance metrics:

$-$ Accuracy

$-$ Precision

$-$ Recall

$-$ F$1$ Score

$-$ ROC$-$AUC Score

$-$ Use cross$-$validation to ensure the reliability of your results.

$-$ Test the performance of your model on the provided test dataset.

D$.$ Comparison and Analysis:

Compare the models based on the performance metrics.

Discuss the strengths and weaknesses of each model in the context of digit

recognition.

Provide insights into the challenges of using machine learning for digit recognition, if any.

Deliverables

A$.$ Report: Submit a detailed report that includes:

$-$ An overview of your data preprocessing steps.

$-$ A brief explanation of the chosen machine learning models.

$-$ Show the performance of your models before and after feature selection based on the metrices specified in the model evaluation section. You must provide a table that shows $5-$fold cross validation performance of the methods you selected to implement.

On separate tables, you must provide the performance comparison of the models on training and test dataset. If features selection results in better performance, you can use selected features to train and test the models.

Discuss the results you obtained and presented in each of the tables above.

B$.$ Code: Submit any code used for data preprocessing, model implementation, and evaluation. Ensure your code is well$-$commented and organized.