1. Write a program implementing multi-layer feed-forward neural networks and training them with back- propagation including momentum. Your program must be able to handle any number of hidden layers and hidden neurons, and should allow the user to specify these at run-time.

2. Randomly choose 4,000 data points from the data files to form a training set, and use the remaining 1,000 data points to form a test set.

3. Train a 1-hidden layer neural network to recognize the digits using the training set. You will probably need a fairly large number of hidden neurons in the range of 100 to 200 and several output neurons. I suggest using 10 output neurons one for each digit such that the correct neuron is required to produce a 1 and the rest 0. To evaluate performance during training, however, you can use target values such as 0.75 and 0.25, as discussed in class. You will probably need hundreds of epochs for learning, so consider using stochastic gradient descent, where only a random subset of the 4,000 points is shown to the network in each epoch. The performance of the network in any epoch is measured by the fraction of correctly classified points in that epoch (Hit-Rate). Save this value at the beginning, and then in every tenth epoch.

4. After the network is trained, test it on the test set. To evaluate performance on the test data, you can use a max-threshold approach, where you consider the output correct if the correct output neuron produces the largest output among all 10 output neurons.