This Assignment requires you to write a computer program that implements the Feed$-$Forward, Back Propagation Algorithm in a multilayer network. This requires you to incorporate the Back Propagation elements into the program you used for Part a$.$ It then requires you to train the network in two ways and then submit computational results. If you do this correctly, your computational answers should match the correct results. Note, as this is a more involved assignment, you will have an extra week in which to complete it$.$ For this assignment, create a neural network program using a language of your choice that implements the feedforward, back propagation $($FFBP$)$ algorithm for a network with the following topology like the one in the lecture: with $2$ input layers x$1$ and x$2,2$ hidden layer h$1$ and h$2$ and $1$ output layer

Develop a neural network program using a language of your choice that implements the feedforward, back propagation $($FFBP$)$ algorithm for a network with the following topology like the one:

Image transcription text

Outout

Develop an implementation of the FFBP algorithm in a neural network from scratch.

Use the following parameters and assumptions:

$$ Set the initial weights for both nodes in the first layer are $0\mathrm{.}3$ and for the second layer as $0\mathrm{.}8.$

$$ Set $\backslash$eta $=1\mathrm{.}0$ and all biases to $0.$

$$ Use inputs $\{1,2\}$ for the two input values and use the desired output value of $0\mathrm{.}7$ as was done in class.

$$ For this part of the assignment, disable routines that update the biases.

Verify that your network performs correctly by checking the results for the first iteration as was done in the lecture videos for this network and shown in the slides $($Hidden layer activations after feedforward: $[0\mathrm{.}71094950\mathrm{.}7109495]$

Output layer activation after feedforward: $0\mathrm{.}757223870792493$

Hidden layer activations after feedforward: $[0\mathrm{.}71094950\mathrm{.}7109495]$

Output layer activation after feedforward: $0\mathrm{.}757223870792493$

Hidden layer activations after feedforward: $[0\mathrm{.}71094950\mathrm{.}7109495]$

Output layer activation after feedforward: $0\mathrm{.}757223870792493$

$).$ Correct any mistakes you find in your code until your answers match those in the slides.

$$ Once you have ensured that your program works correctly, do the following exercise after you re$-$enable routines that update the bias:

$$ Train the network with the following two input$/$output pairs using the two different approaches described below and the initial weights, bias and $\backslash$eta values noted above:

$(1,1)->0\mathrm{.}9$; $(-1,-1)->0\mathrm{.}05$;

$$ Before you run your code to perform each of the methods described below, make sure you use the proper initial values for the weights and biases as noted above, otherwise your final computations will be off.

Method $1$:

For each cycle of this training procedure, present the first input$/$output pair, perform the back propagation technique to update the weights, then present the second input$/$output pair and again perform the back propagation technique to update the weights. This constitutes a single cycle. Perform $15$ such cycles and determine the errors E for each input$/$output pair. This method essentially updates the weights by alternately presenting each input$/$output pair $...$ the first pair, and then the second pair and so on$.$ After the $15$th training cycle, present the input values to the network and print out the total Error $($Big E$)$ and the final weights associated with each input$/$output pair.

Method $2$:

In this method, we update weights for one input$/$output pair for $15$ iterations of the FFBP algorithm, then present the second input$/$output pair and run the FFBP algorithm to update weights for another $15$ iterations. Thus, for each cycle, present the first input$/$output pair, run the FFBP for $15$ iterations, then present the second input$/$output pair and run the FFBP for another $15$ iterations. This second set of iterations therefore begins updating the weights from the values obtained after the first $15$ iterations with the first input$/$output pair. After the training of the second input$/$output pair, present the input of the first pair, print out the total Error $($Big E$)$ and the final weights associated with it$,$ then present the input of the second pair and print out the total

Question:

$1.$ After training the network with Method $1$ :

a$)$ What is the output of the network $($the activation value of the output node$)$ when the network is presented with an input of $[1\mathrm{.}0,1\mathrm{.}0]?$ Answer to $6$ significant decimal digits.

b$)$ What is the value of Big E at the end of the training when the network is presented with an input of $[1\mathrm{.}0,1\mathrm{.}0]?$ Answer to $6$ significant decimal digits.

c$)$ What is the output value when the network is presented with an input of $[-1\mathrm{.}0,-1\mathrm{.}0]?$ Answer to $6$ significant decimal digits.

d$)$ What is the value of Big E when the network is presented with the input $[-1\mathrm{.}0,-1\mathrm{.}0]?$ Answer to $6$ significant decimal digits.

$2.$ After training the network with Method $2,$

a$)$ What is the network output when presented with input $[1\mathrm{.}0,1\mathrm{.}0]?$ Answer to $6$ signi