Testing PCAMotivation

$$ Bookmark this page

Exercises due Jul $17,202417$:$29$ IST

Review: True or False

$2$ points possible $($graded$)$

Consider the classification decision rule

hat$(y)=$sign$(*(x))$

where $xin{R}^d$ represent input data and yin$\{1,-1\}$ is the corresponding predicted labels, and we have

omitted the bias$/$offset term for simplicity.

Given the model above, determine if the following statements are True or False.

The feature map $$ is function from $R^d$ to $R^d.$

If $(x)in{R}^D,$ then the classification parameter $$ is also a vector in $R^D.($Answer based on the model

as written.$)$

$0\mathrm{.}0/1\mathrm{.}0$ point $($graded$)$

Use

to compute a $18-$dimensional PCA representation of the MNIST training and test

datasets, as illustrated in

Retrain your softmax regression model $($using the original labels$)$ on the MNIST training dataset and report its

error on the test data, this time using these $18-$dimensional PCA$-$representations rather than the raw pixel

values.

If your PCA implementation is correct, the model should perform nearly as well when only given $18$ numbers

encoding each image as compared to the $784$ in the original data $($error on the test set using PCA features

should be around $0\mathrm{.}15).$ This is because PCA ensures these $18$ feature values capture the maximal amount of

variation from the original $784-$dimensional data.

Error rate for $18-$dimensional PCA features $=$