An Autoencoder is designed to compress a high$-$dimensional input down to a small number of dimensions in such a way that all $($or most$)$ of the information in the original input can be recovered. These models are composed of two networks: a compression network and a decompression network. The two networks are inverses of each other. This means that if$,$ for example, the compression network was an MLP model with three layers with $(100,40,20)$ neurons, the decompression network would have three layers with $(20,40,100)$ neurons.

Note: These models do not use labels and are, hence, unsupervised.

An input is presented to the compression network, the compressed output is presented to the decompression network, the decompressed output is compared against the original input to calculate the $$reconstruction error$$ and the that error is used in a loss function to train the compression and decompression networks in tandem$.$

For training this type of model, hold all of digit $3$ and some fraction of every other digit for your test set.

Experiment with a few different structures for your autoencoders. Build a few whose performance you are pleased with. Finally, measure the performance of these models on the test set.

Since the test set contains a digit that the models has never seen before, how well do you expect the models to perform, in terms of reconstruction error, on this test set? Did the models perform in line with your expectation? Explain why. What does this imply for handling, or identifying, previously unseen classes of data?