Using sigmoid function,

$[4$ pts$]$ explain why large initial values of $w$ cause a

vanishing gradient

$[4$ pts$]$ Using the following formula, explain why the

sigmoid function causes a vanishing gradient in multilayer

$\frac{\text{d}\text{e}\text{l}\text{J}()}{\text{d}\text{e}\text{l}{}_1}=\frac{\text{d}\text{e}\text{l}\text{J}()}{\text{d}\text{e}\text{l}(hat(y))}**\frac{\text{d}\text{e}\text{l}(hat(y))}{del{z}_l}**\frac{del{z}_l}{del{z}_{l-1}}**dots**\frac{del{z}_1}{\text{d}\text{e}\text{l}{}_1}$

networks.

With the following neural network, $($Assume each layer has the same

number of nodes $($width$)$ and nodes are fully connected$)$

$[4$ pts$]$ Show the formula for the $($approximate$)$ total number of parameters

in this neural network. Explain the formula. $($p$.12$ in slides$)$

$[4$ pts$]$ Based on q$.1),$ explain why deep layer network can reduce the total

number of parameters compared to shallow layer network.

$[4$ pts$]$ Explain why deep layer networks are efficient in feature learning.

$[4$ pts$]$ Explain the role of 'callback' in the following Keras program code.

callback $=$ keras.callbacks.EarlyStopping$($monitor$=$'loss', patience$=3)$

history $=$ model.fit$($np$.$arange$(100).$reshape$(5,20),$ np$.$zeros$(5),$ epochs$=10,$ batch$\_$size$=1,$

callbacks$=[$callback$],$ verbose $=0)$