A $1-$D convolutional neural network has three convolutional layers followed by a softmax

unit. The convolutional layers have the following structure:

The first hidden layer has $2$ filters of kernel$-$width $2$ ;

The second layer has $3$ filters of kernel$-$width $3$;

The third layer has $2$ filters of kernel$-$width $4.$

Assume that the stride of the convolution is $1$ in every layer. As explained in class, the

convolution layers of this TDNN are exactly equivalent to scanning the input with a $($shared$-$

parameter$)$ MLP $($and passing the set of outputs of the MLP at the individual time instants

through a final softmax$).$ What would be the architecture of this scanning MLP$?$

Hint: Piazza @$659$

A three layer MLP with $4$ neurons in the first layer, $9$ neurons in the second layer and $8$ neurons in the

third layer

A one$-$layer MLP with a layer of size $8$

A three layer MLP with $2$ neurons in the first layer, $3$ in the second layer and $2$ in the third layer

A three$-$layer MLP with $12$ neurons in the first layer, $12$ in the second layer and $2$ in the third layer