The structure of a Neural Network is given below. W W Y H X1 X2 H...

Transcribed Image Text:

The structure of a Neural Network is given below. W W Y H X1 X2 H The input value X = X2 X-8 and the desired output value is Y The initial weight matrix for the first layer W: w = [WW] = [36] 4 (1) The bias weight matrix W 1 The initial weight matrix for the second layer W2: w = [ww] = 2 3 4 3 (2) The bias weight matrix W = [] Perform the following tasks: Forward propagation: calculate the value of the hidden nodes using sigmoid function as the activation function, and obtain the predicted value for output nodes using softmax as the output function. Com- pute the gradient of the output VY using cross-entropy loss function. Back propagation: Given VY, obtain the gradients for the weight matrix and bias vector for each layer. Update the weight matrix and bias vectors for each layer with their estimated gradients, using a learning rate of 0.5, and then compute the output value using the updated weight matrices. Verify that the updated weight matrices have reduced the output loss function, as compared to those obtained using the initial weight matrices For each task above, write out its equations and the estimated values. The structure of a Neural Network is given below. W W Y H X1 X2 H The input value X = X2 X-8 and the desired output value is Y The initial weight matrix for the first layer W: w = [WW] = [36] 4 (1) The bias weight matrix W 1 The initial weight matrix for the second layer W2: w = [ww] = 2 3 4 3 (2) The bias weight matrix W = [] Perform the following tasks: Forward propagation: calculate the value of the hidden nodes using sigmoid function as the activation function, and obtain the predicted value for output nodes using softmax as the output function. Com- pute the gradient of the output VY using cross-entropy loss function. Back propagation: Given VY, obtain the gradients for the weight matrix and bias vector for each layer. Update the weight matrix and bias vectors for each layer with their estimated gradients, using a learning rate of 0.5, and then compute the output value using the updated weight matrices. Verify that the updated weight matrices have reduced the output loss function, as compared to those obtained using the initial weight matrices For each task above, write out its equations and the estimated values.