Deep Learning:

Scalar computation graph: (a) Draw the computation graph for a regression network with three hidden units and one output unit. Assume logistic activation at hidden layer nodes and linear activation at output. You may use logistic nodes without having to break them up into elementary computations. Assume L2 loss. (b) Assume the current weight parameters guess is given by: w1w2w3v=(0.01,0.02,0.03)=(0.03,0.01,0.02)=(0.02,0.03,0.01)=(0.01,0.02,0.03,0.04) where w1,w2,w3 are hidden unit weight vectors and v is the output unit weight vector. Assume the training data in the batch is given by: x1,y1=[(1,2),8]x2,y2=[(1,3),11]x3,y3=[(2,2),10] Compute the values at the network for the forward pass for each data point. (c) Compute the gradients of the loss function with respect to w1,w2,w3,v due to all training examples using backpropagation. (d) Compute the gradients using the formula given in class and compare to your previous result. Scalar computation graph: (a) Draw the computation graph for a regression network with three hidden units and one output unit. Assume logistic activation at hidden layer nodes and linear activation at output. You may use logistic nodes without having to break them up into elementary computations. Assume L2 loss. (b) Assume the current weight parameters guess is given by: w1w2w3v=(0.01,0.02,0.03)=(0.03,0.01,0.02)=(0.02,0.03,0.01)=(0.01,0.02,0.03,0.04) where w1,w2,w3 are hidden unit weight vectors and v is the output unit weight vector. Assume the training data in the batch is given by: x1,y1=[(1,2),8]x2,y2=[(1,3),11]x3,y3=[(2,2),10] Compute the values at the network for the forward pass for each data point. (c) Compute the gradients of the loss function with respect to w1,w2,w3,v due to all training examples using backpropagation. (d) Compute the gradients using the formula given in class and compare to your previous result