Question

Consider a CNN that takes in $\backslash (32\backslash$times $32\backslash )$ grayscale images and has a single convolution layer with three $\backslash (5\backslash$times $5\backslash )$ convolution filters $($without boundary padding$).$

$($a$)$ Draw a sketch of the input and first hidden layer similar to the one below.

FIGURE $10\mathrm{.}8.$ Architecture of a deep CNN for the CIFAR$100$ classification task. Convolution layers are interspersed with $\backslash (2\backslash$times $2\backslash )$ max$-$pool layers, which reduce the size by a factor of $2$ in both dimensions.

$($b$)$ How many parameters are in this model?

$($c$)$ Explain how this model can be thought of as an ordinary feedforward neural network with the individual pixels as inputs, and with constraints on the weights in the hidden units. What are the constraints?

$($d$)$ If there were no constraints, then how many weights would there be in the ordinary feed$-$forward neural network in $($c$)?$