Question: need mathematical solution 3. [3 marks] Deep learning (a specific subclass of machine learning methods) consists of constructing very deep neural networks with many layers,

need mathematical solution 3. [3 marks] Deep

need mathematical solution

3. [3 marks] Deep learning (a specific subclass of machine learning methods) consists of constructing very deep neural networks with many layers, sometimes over 1,000. These networks can have large numbers of parameters and become very expensive to train. One surprising way to reduce the number of parameters in the network is to share parameters between layers. That is, each layer uses the same weight matrix and bias parameters. Derive a new version of the backpropogation algorithm which would allow you to compute the gradient of the objective E(W, b) = lly, F(x)) with respect to the parameters W and b where l(y,) is some loss function and F(x) = fz(fl-16--(f1(x))...)) is a neural network with layers fi(x) = P(Wx + b) where W and b are the same parameters for each layer. Note that the input dimensions, output dimension and width (number of hidden units in each layer) are all equal in this simplified setup

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related General Management Questions!