Question: (a) Suggest a network architecture appropriate for MNIST and explain why the crossentropy cost function might be preferable to the mean-squared-error when the sigmoid activation

 (a) Suggest a network architecture appropriate for MNIST and explain why

(a) Suggest a network architecture appropriate for MNIST and explain why the crossentropy cost function might be preferable to the mean-squared-error when the sigmoid activation function is used. (5 marks) (b) Explain the problem of overtraining a neural network and how it could be alleviated. How does MNIST address this problem? (5 marks) (c) Briefly explain what gradient descent means and how it relates to the cost or loss function. Explain what each of the following symbols represents and the relationship between then in gradient descent. wijlandwijlC? (d) At the core of the backpropagation algorithm are the matrix formulae: wjklC=jlakl1L=(aLy)(zL)l=((wl+1)Tl+1)(zl) Briefly explain what each term means and which one in particular relates to backpropagation. Also, for a three layer network, write a Python code fragment which includes each of these formulae. (10 marks)

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!