Assignment

Assignment: Cross$-$Validation and Hyperparameter Optimization with MNIST Dataset

Assignment Topic

In this assignment, students will use Scikit$-$Learn to build a classifier for the MNIST dataset and enhance its performance using Cross$-$Validation, RandomizedSearchCV, and GridSearchCV. Students are expected to document their work in a detailed Python Notebook.

Assignment Steps

Load the MNIST Dataset:

Load the MNIST dataset using Scikit$-$Learn or another suitable library.Split the data into training and test sets.

Explore the Data:

Visualize sample images from the dataset $($e$.$g$.,$ a few example images and their labels$).$Provide information about the dataset dimensions.

Define the Model:

Use a Support Vector Machine $($SVM$)$ classifier as the model.

Apply Cross$-$Validation:

Use $5-$fold Cross$-$Validation to calculate the baseline accuracy score of the model.Report the minimum and maximum accuracy scores obtained during Cross$-$Validation.

Hyperparameter Optimization with RandomizedSearchCV:

Apply RandomizedSearchCV with the following hyperparameter ranges for the SVM model:

C: $[0\mathrm{.}1,1,10,100]$gamma: $[\text{'}$scale$\text{'},$ 'auto'$]$kernel: $[\text{'}$linear$\text{'},\text{'}$rbf$\text{'},$ 'poly'$]$

Report the best hyperparameters and the corresponding model score.

Hyperparameter Optimization with GridSearchCV:

Use a narrower range of hyperparameters $($based on the RandomizedSearchCV results$)$ and apply GridSearchCV.Report the best hyperparameters and the corresponding model score.

Compare Results:

Compare the baseline model, RandomizedSearchCV, and GridSearchCV results.Visualize the results in a table or chart.

Evaluate the Best Model on the Test Set:

Use the best$-$performing model to evaluate accuracy on the test set.Visualize the confusion matrix and provide a brief interpretation.

Deliverables

Colab Notebook File:

Students must create their assignment using Google Colab.Write clean and well$-$commented code.Include relevant visualizations and text explanations.

Colab Link Submission:

Share the Google Colab link along with appropriate permissions to access it$.$

Results and Discussion:

Provide a brief discussion on model performance.Highlight the impact of Cross$-$Validation and hyperparameter optimization.Report the minimum and maximum accuracy scores obtained during Cross$-$Validation.

$$

Evaluation Criteria

Criteria

Points

Dataset loading and exploration

$10$

Cross$-$Validation implementation

$20$

Use of RandomizedSearchCV

$20$

Use of GridSearchCV

$20$

Visualization of results

$10$

Code quality and explanations

$10$

Results and discussion

$10$

Total

$100$