Multi-layer Neural Network

1. Copy your previous python file mysvm.py as mymlnn.py. Add the following packages to the existing list. (Use the same dataset as mysvm.py)

import neurolab as nl

2. Use ''' ''' pair to comment all lines that follow after the below lines.

X = X_encoded[:, :-1].astype(int)

y = X_encoded[:, -1].astype(int)

3. Following the above instructions, add the below lines to the python file. Run the program and show the printout.

y = np.expand_dims(y, axis=1)

minmax_X = []

X1, y1 = X.tolist(), y.tolist();

for i,item in enumerate(X[0]):

minmax_X.append([X[:,i].min(),X[:,i].max()])

print(minmax_X)

4. Add the following lines to use Gradient Descent method for training a three-layer network. (1) what is nl.net.newff for? (2) what is [10, 6, 1] for?

nn = nl.net.newff(minmax_X, [10, 6, 1])

nn.trainf = nl.train.train_gd

error_progress = nn.train(X1, y1, epochs=50, goal=0.01)

5. Add the following lines and show the figure (paste it below).

plt.figure()

plt.plot(error_progress)

plt.xlabel('Epochs')

plt.ylabel('Error')

plt.title('Training progress')

plt.show()

6. (1) Replace [10, 6, 1] with[15, 6, 1] and show the figure. (2) Replace epochs=50 with epochs=10 and show the figure. (3) Try different numbers and find out what the best parameters are fitting your program.

7. Add the following lines. Note that they are same as those in the original mysvm.py file.

input_data = [['52', 'Self-emp-not-inc', '209642', 'HS-grad', '9', 'Married-civ-spouse', 'Exec-managerial', 'Husband', 'White', 'Male', '0', '0', '45', 'United-States'], \

['42', 'Private', '159449', 'Bachelors', '13', 'Married-civ-spouse', 'Exec-managerial', 'Husband', 'White', 'Male', '5178', '0', '40', 'United-States']]

input_data = np.array(input_data)

input_data_encoded = np.empty(input_data.shape)

count = 0

for i, item in enumerate(input_data[0]):

if item.isdigit():

input_data_encoded[:,i] = input_data[:,i]

else:

input_data_encoded[:,i] = label_encoder[count].transform(input_data[:,i])

count += 1

input_data_encoded = input_data_encoded.astype(int)

print(input_data_encoded)

8. Add the following lines and run the program. Show the printout.

for t in input_data_encoded.tolist():

predicted_class = nn.sim([t])[0]

print(t, '-->', predicted_class)

9. Consider to improve this program by using model_selection.train_test_split to split the training (80%) and the testing (20%) data sets. Please (1) show your code, and (2) show the training progress figure.

10. Consider to calculate the accuracy. Please (1) show your code, and (2) show the training progress figure. Hint: predicted_y1 and y1.