a$).$ Consider the following neural network architecture with one hidden unit and one output unit as follows:

$[4$ Marks$]$

Assume a sequence length of size $n$ with each xi represented by a $m-$dimensional vector. Let $h$ represent a single hidden layer with $H$

hidden units and tanh activation. Model predicts y using softmax function with $U$ representing the weight matrix and $d$ representing the

bias for node $y.$

What are the dimensions of W$,$ U$,$ b$,$ h$,$ d and the input matrix $x.$

ii$)$ What are the total number of parameters in the above architecture?

b$).$ Two friends Mr Raju and Mr Robert, while experimenting with a deep neural network found that their model is over$-$fitting the given

training data. The friends referred to the available literature and materials and came up with a solution to reduce the over$-$fitting. Mr Raju

decided to add $1$ drop out along with batch normalization to his model. Whereas Mr Robert decided to go with a deeper model hence he

added more layers. But he found that his model is struggling to train because of his hardware limitations$.$ So he decided to employ early

stopping. Assess whether the decisions Mr Raju and Mr Ravi are taking helps them to proceed in the night direction.

$[4$ Marks$]$