The article introduces some simplifying notation to avoid clutter in the examples, but it is also...

Transcribed Image Text:

The article introduces some simplifying notation to avoid clutter in the examples, but it is also good to cut through their abstraction and be clear exactly what the underlying functions, weights and bias parameters are. In the picture below, we show a neural network on the left, with input value x going to 4 neurons wix+bi, i-1,2,3,4, with nonlinear sigmoid activations. In turn, the 4 outputs y1.y2y3.y4 from this hidden layer pass to an output neuron that forms a simple linear weighted combination of their values (without a sigmoid, and without a bias value), introducing four more weights w5w6w7w8. So the network has 8 weights and 4 bias values that need to be set. or 12 parameters total. The questions that follow ask how to set these parameter values if we want the output from this network to be the function defined and plotted on the right of the picture. y = o(w, x + b) y = o(wx + b) y = o(wx + b) y = o(wx + b) Let w, w=w, w = 1000 Let |b||b||b||b| Yout WsY1 W6Y2 + WyY3+ W8Y4 1.5 1. 5 -5 -1- Yout -0.5 for 1x .5 1.0 for .6x .8 otherwise 0 Because this question will be automatically graded by canvas, I need to impose constraints that will make the values and order of your answers unique. First, assume the first 4 weights w1,w2.w3.w4 are set to value 1000, you do not get to choose those. Second, when entering your answers for the 4 bias parameters, assume the magnitudes are in sorted order. That is, absolute value of b1 will be smaller than absolute value of b2, and so on, with absolute value of b4 being largest. I believe those two constraints are enough to also uniquely determine the values you must enter for w5w6w7w8. In any case, don't try to get overly creative, just enter the obvious answers in the obvious orders based on the approximation procedure presented in the article. The article introduces some simplifying notation to avoid clutter in the examples, but it is also good to cut through their abstraction and be clear exactly what the underlying functions, weights and bias parameters are. In the picture below, we show a neural network on the left, with input value x going to 4 neurons wix+bi, i-1,2,3,4, with nonlinear sigmoid activations. In turn, the 4 outputs y1.y2y3.y4 from this hidden layer pass to an output neuron that forms a simple linear weighted combination of their values (without a sigmoid, and without a bias value), introducing four more weights w5w6w7w8. So the network has 8 weights and 4 bias values that need to be set. or 12 parameters total. The questions that follow ask how to set these parameter values if we want the output from this network to be the function defined and plotted on the right of the picture. y = o(w, x + b) y = o(wx + b) y = o(wx + b) y = o(wx + b) Let w, w=w, w = 1000 Let |b||b||b||b| Yout WsY1 W6Y2 + WyY3+ W8Y4 1.5 1. 5 -5 -1- Yout -0.5 for 1x .5 1.0 for .6x .8 otherwise 0 Because this question will be automatically graded by canvas, I need to impose constraints that will make the values and order of your answers unique. First, assume the first 4 weights w1,w2.w3.w4 are set to value 1000, you do not get to choose those. Second, when entering your answers for the 4 bias parameters, assume the magnitudes are in sorted order. That is, absolute value of b1 will be smaller than absolute value of b2, and so on, with absolute value of b4 being largest. I believe those two constraints are enough to also uniquely determine the values you must enter for w5w6w7w8. In any case, don't try to get overly creative, just enter the obvious answers in the obvious orders based on the approximation procedure presented in the article.