Assume we are training a neural network with two inputs and for the OR gate. The...

Transcribed Image Text:

Assume we are training a neural network with two inputs and for the OR gate. The table below represents the input and expected output for the OR gate: i 1 2 3 4 1 2 0 0 0 1 0 1 output y 0 1 1 1 We will create a neural network with two inputs, one output, and no hidden layer. The activation function will be the sigmoid function S(x) = 1 1+ e-z' x1 X2 W W2 = 0.4 and = -0.2, and initial bias b = 0.1. Begin with the initial weights w = The observed output is calculated as = S(s) where s = wx1 +wx2 + b. If we were to implement this in a practical example, we know that the output is expected as either 0 or 1, so we could simply round to the nearest integer afterwards. (a) For each of the four inputs (1, 2) in the table above, determine the observed output for the initial weights and bias. (b) Using the mean squared error function error = i=1 (i - y) calculate the error observed in part (a). (c) Using the chain rule, calculate the following derivatives derror wi Jerror w derror b After the 100 epochs, what are the values s and for each input? What are the values of w, W2, b? (e) Did the observed output adjust properly to mimic the expected output? Note: For those familiar with neural networks, this training simulation has quite a few issues with it. For instance, we are overfitting the data since our actual data is the training data. This is due to the small data amounts. Another issue is we are not randomizing the order of the inputs when training. All of this (and more!) will be discussed in your future ML & AI courses. Assume we are training a neural network with two inputs and for the OR gate. The table below represents the input and expected output for the OR gate: i 1 2 3 4 1 2 0 0 0 1 0 1 output y 0 1 1 1 We will create a neural network with two inputs, one output, and no hidden layer. The activation function will be the sigmoid function S(x) = 1 1+ e-z' x1 X2 W W2 = 0.4 and = -0.2, and initial bias b = 0.1. Begin with the initial weights w = The observed output is calculated as = S(s) where s = wx1 +wx2 + b. If we were to implement this in a practical example, we know that the output is expected as either 0 or 1, so we could simply round to the nearest integer afterwards. (a) For each of the four inputs (1, 2) in the table above, determine the observed output for the initial weights and bias. (b) Using the mean squared error function error = i=1 (i - y) calculate the error observed in part (a). (c) Using the chain rule, calculate the following derivatives derror wi Jerror w derror b After the 100 epochs, what are the values s and for each input? What are the values of w, W2, b? (e) Did the observed output adjust properly to mimic the expected output? Note: For those familiar with neural networks, this training simulation has quite a few issues with it. For instance, we are overfitting the data since our actual data is the training data. This is due to the small data amounts. Another issue is we are not randomizing the order of the inputs when training. All of this (and more!) will be discussed in your future ML & AI courses.

Answer rating: 100% (QA)

Solutions Step 1 This is the complete answer to this with a very detailed explanation This is the complete answer for part A in this we need to calculate the observed output for each of the four input... View the full answer