(a) Suggest a network architecture appropriate for MNIST and explain why the cross- entropy cost function...

   

Transcribed Image Text:

(a) Suggest a network architecture appropriate for MNIST and explain why the cross- entropy cost function might be preferable to the mean-squared-error when the sigmoid activation function is used. (5 marks) (b) Explain the problem of overtraining a neural network and how it could be alleviated. How does MNIST address this problem? (5 marks) (c) Briefly explain what gradient descent means and how it relates to the cost or loss function. Explain what each of the following symbols represents and the relationship between then in gradient descent. w, and ac aw Vij ? (d) At the core of the backpropagation algorithm are the matrix formulae: ac = 8} ak awk 1-1 s = (a - y) o' (z) 8 = ((w+1) 8+1) o' (2) Briefly explain what each term means and which one in particular relates to backpropagation. (5 marks) Also, for a three layer network, write a Python code fragment which includes each of these formulae. (10 marks) (a) Suggest a network architecture appropriate for MNIST and explain why the cross- entropy cost function might be preferable to the mean-squared-error when the sigmoid activation function is used. (5 marks) (b) Explain the problem of overtraining a neural network and how it could be alleviated. How does MNIST address this problem? (5 marks) (c) Briefly explain what gradient descent means and how it relates to the cost or loss function. Explain what each of the following symbols represents and the relationship between then in gradient descent. w, and ac aw Vij ? (d) At the core of the backpropagation algorithm are the matrix formulae: ac = 8} ak awk 1-1 s = (a - y) o' (z) 8 = ((w+1) 8+1) o' (2) Briefly explain what each term means and which one in particular relates to backpropagation. (5 marks) Also, for a three layer network, write a Python code fragment which includes each of these formulae. (10 marks)

Answer rating: 100% (QA)

a For MNIST a straightforward and successful organization design could be a feedforward brain networ... View the full answer