Feedforward Neural Networks $(25$pt$)$

Cuonsider two models: $($a$)$ a $2-$layer feedforward neural network $($i$.$e$.,1$ hidden layer with

f$($x$,$ W$1,$ b$1,$ W$2,$ b$2)=$ W$2$ max$(0,$ W$1$x $+$ b$1)+$ b$2),$ and $($b$)$ same as before but with leaky

ReLU $($f$($x$)=$ x if x $>0,$ else f$($x$)=0\mathrm{.}01$ x$).$

$1.$ Build the above classifiers using Keras and Tensorflow and solve the classification

problem for MNIST$/$Fashion MNIST $($link$).$

$2.$ Discuss how optimizer choice influences performance.

$3.$ What happens when the number of hidden units chosen is much smaller. Similarly,

what happens when the number of hidden units chosen is much higher?