Datasets :

The MNIST database is developed by Yann LeCunn, Corinna Cortes and Christopher J.C. Burges. It contains handwritten digits and has a training set of 60,000 examples, and a test set of 10,000 examples. Each image is 28 x 28 in size. In this project, the training data set in the filename train.csv, has 785 columns. The first column, called "label", is the digit that was drawn by the user. The rest of the columns contain the pixel-values of the associated image. The test data set in the filename test.csv, is the same as the training set, except that it does not contain the "label" column. Neural Network:

Using this dataset, you are required to implement a back-propagation neural network to classify the digits. You are only allowed to use libraries for linear algebra operations (matrix multiplication, inversion etc). You are not allowed to use any existing machine learning or statistic toolkits/ libraries for this project.

For simplicity, use only one hidden layer and investigate various network structures (choose 3 different number of hidden nodes per layer at different scale. For an example, 10, 1000, and 10000) and report the performance of the neural networks.

You may start your program by completing the given codes in the ANSWER SHEET.

#loading the dataset

train_data = pd.read_csv("../train.csv") #loading the training dataset from folder

test_data= pd.read_csv("../test.csv")#loading the testing dataset from folder

#separating labels and pixels

train_labels=np.array(train_data.loc[:,'label'])

train_data=np.array(train_data.loc[:,train_data.columns!='label'])

#converting train_label in one hot encoder representation

train_data=np.reshape(train_data,[784,42000])

train_label=np.zeros((10,42000))

for col in range (42000):

val=train_labels[col]

for row in range (10):

if (val==row):

train_label[val,col]=1

print("train_data shape="+str(np.shape(train_data)))

print("train_label shape="+str(np.shape(train_label)))

# activation function sigmoid

def sigmoid(Z):

A = 1/(1+np.exp(-Z))

cache = Z

return A, cache

# activation function softmax

def softmax(Z):

e_x = np.exp(Z)

A= e_x / np.sum(np.exp(Z))

cache=Z

return A,cache

# weights and bias initialize code here

# feed forward neural network code here

# back propagation and weight updates code here.

# testing phase here

# performance analysis