Please solve all four questionsConsider a feedforward neural network that performs regression on a $p-$dimensional input to produce a scalar output. It has $m$ hidden layers and

each of these layers has $k$ hidden units. What is the total number of trainable parameters in the network? Ignore the bias terms.

$pk+m{k}^2$

$pk+m{k}^2+k$

$pk+(m-1)k^2+k$

$p^2+(m-1)pk+k$

$p^2+(m-1)pk+k^2$

Consider a neural network layer defined as $y=$ReLU$(Wx).$ Here $xin{R}^p$ is the input, $yin{R}^d$ is the output and $Win{R}^{dp}$ is the parameter

matrix. The ReLU activation $($defined as ReLU$(z)$:$=$max$(0,z)$ for a scalar $z)$ is applied element$-$wise to $Wx.$ Find $\frac{del{y}_1}{del{W}_{ij}}$ where $i=1,$dots,$d$ and

$j=1,$dots,$p.$ In the following options, I $($condition$)$ is an indicator function that returns $1$ if the condition is true and $0$ if it is false.

$I(y_i>0)x_i$

$I(y_i>0)x_j$

$I(y_{i}0)x_i$

$I(y_i>0)W_{ij}{x}_j$

$I(y_{i}0)W_{ij}{x}_i$

Consider a two$-$layered neural network $y=(W^{(B)}(W^{(A)}x)).$ Let $h=(W^{(A)}x)$ denote the hidden layer representation. $W^{(A)}$ and

$W^{(B)}$ are arbitrary weights. Which of the following statement$($s$)$ is$/$are true? Note: $gra{d}_g(f)$ denotes the gradient of $f$ w$.$r$.$t $g.$

$gra{d}_h(y)$ depends $on{W}^{(A)}.$

$gra{d}_{W^{(A)}}(y)$ depends $on{W}^{(B)}.$

$gra{d}_{W^{(A)}}(h)$ depends $on{W}^{(B)}.$

$gra{d}_{W^{(B)}}(y)$ depends $on{W}^{(A)}.$

Which of the following statement$($s$)$ about the initialization of neural network weights is$/$are true?

Two different initializations of the same network could converge to different minima.

For a given initialization, gradient descent will converge to the same minima irrespective of the learning rate.

The weights should be initialized to a constant value.

The initial values of the weights should be sampled from a probability distribution.