Prove that the PCA is the best linear method for transformation $($with orthonormal

bases$)$II$.$ Feature Extraction $($for dataset B$)$

Use PCA as a dimensionality reduction technique to the data, compute the eigenvectors

and eigenvalues.

Plot a $2-$dimensional representation of the data points based on the first and second

principal components. Explain the results versus the known classes $($display data points

of each class with a different color$).$

Repeat step $2$ for the $5$th and $6$st components. Comment on the result.

Use the Naive Bayes classifier to classify $8$ sets of dimensionality reduced data $($using the

first $2,4,10,30,60,200,500,$ and all $784$ PCA components$).$ Plot the classification error

for the $8$ sets against the retained variance of each case.

As the class labels are already known, you can use the Linear Discriminant Analysis $($LDA$)$

to reduce the dimensionality, plot the data points using the first $2$ LDA components

$($display data points of each class with a different color$).$ Explain the results obtained in

terms of the known classes. Compare with the results obtained by using PCA.

Prove that the PCA is the best linear method for transformation $($with orthonormal

bases$)$q$6$