Consider the following network structure. You can assume the initial weights. Assume bias to be zero for easier computations. Given that $<$ x$1,$ x$2,$y$1,$y$2>=<1,1,0,1>$ where $$y is the target. Assume $\backslash$beta $=0\mathrm{.}9$ and $\backslash$eta $=0\mathrm{.}01.$

x$1====>$h$1====>$y$1$

x$1====>$h$2====>$y$1$

x$1====>$h$1====>$y$2$

x$2====>$h$1====>$y$2$

x$2====>$h$2====>$y$1$

x$2====>$h$2====>$y$2$

$($a$)$ Compute the forward propagation and generate the output. Use Relu for hidden layers and Sigmoid activation function for output layer.

$($b$)$ Compute the Softmax loss function for both outputs.

$($c$)$ Let the initial weights that assumed be the weights $[$at time $($t$-1).$ Compute the weights v$21,$ w$12$ and w$22$ at time t using SGD$.$

$($d$)$ Let the weight at time t be the ones computed in part $($c$).$ Compute the weights v$21,$ w$12$ and w$22$ at $($t $+1)$ when momentum is used.