Write a program in Python for implementing multi-layer feed-forward neural networks and training them with back-propagation including momentum. You should be able to choose the number of hidden layers, number of hidden neurons in each layer, number of output neurons, learning rate, momentum, etc. You have to also be able to set the learning rate and momentum separately for each layer of weights. Adaptation of learning rate is optional.

The program should be able to use the MNIST data set and do the following:

1. Randomly choose 4,000 data points from the data files to form a training set, and use the remaining 1,000 data points to form a test set.

2. Train a 1-hidden layer neural network to recognize the digits using the training set. You will probably need a fairly large number of hidden neurons (in the range of 100 to 200, but you can try fewer) and several output neurons. Suggestion: using 10 output neurons (one for each digit) such that the correct neuron is required to produce a 1 and the rest 0. To evaluate performance during training, however, you can use lower thresholds, such as 0.75 and 0.25. You will probably need hundreds of epochs for learning, so consider using stochastic gradient descent, where only a random subset of the 4,000 points is shown to the network in each epoch. The performance of the network in any epoch is measured by the fraction of correctly classified points in that epoch. Save this value at the beginning, and then in every tenth epoch.

3. After the network is trained, test it on the test set. To evaluate performance on the test data, you can use a soft-max approach, where you consider the output correct if the correct output neuron produces the largest output among all 10 output neurons (even if that output is not above 0.75 or 0.5)