Question $1$

Given the following neural network with fully connection layer and ReLU activations, including two input units $($i$1,$ i$2),$ four hidden units $(h1,h2)$ and $(h3,h4.$ The output units are indicated as $(o1,o2)$ and their targets are indicated as $(t1,t2).$ The weights and bias of fully connected layer are called $w$ and $b$ with specific sub$-$descriptors.

The values of variables are given in the following table:

$\backslash$table$[[$Variable$,$i$\_(1),$i$\_(2),$w$\_(11),$w$\_(12),$w$\_(21),$w$\_(22),$w$\_(31),$w$\_(32),$w$\_(41),$w$\_(42),$b$\_(1),$b$\_(2),$b$\_(3),$b$\_(4),$t$\_(1),$t$\_(2)],[$Value$,2\mathrm{.}0,-1\mathrm{.}0,1\mathrm{.}0,-0\mathrm{.}5,0\mathrm{.}5,-1\mathrm{.}0,0\mathrm{.}5,-1\mathrm{.}0,-0\mathrm{.}5,1\mathrm{.}0,0\mathrm{.}5,-0\mathrm{.}5,-1\mathrm{.}0,0\mathrm{.}5,1\mathrm{.}0,0\mathrm{.}5]]$

$($a$)$ Compute the output $(01,02)$ with the input $(i1,i2)$ and network parameters as specified above. Write down all calculations, including intermediate layer results.

$($b$)$ Compute the mean squared error of the output $(o1,o2)$ calculated above and the target $(t1,t2.$

$($c$)$ Update the wight w$21$ using gradient descent with learning rate $0\mathrm{.}1$ as well as the loss computed previously. $($Please write down all your computations.$)$Given the following neural network with fully connection layer and ReLU activations, including two input units $($i$1,$ i$2),$ four hidden units $($h$1,$ h$2)$ and $($h$3,$ h$4).$ The output units are indicated as $($o$1,$ o$2)$ and their targets are indicated as $($t$1,$ t$2).$ The weights and bias of fully connected layer are called w and b with speci$$c sub$-$descriptors