Consider the following network structure. You can assume the initial weights. Assume bias

to be zero for easier computations. Given that $<$ x$1,$ x$2,$y$1,$y$2>=<1,1,0,1>$ where $$y is

the target. Assume $=0\mathrm{.}9$ and $=0\mathrm{.}01.$

$2$

x$1$

x$2$

h$1$

h$2$

$$y$1$

$$y$2$

w$11$

w$12$

w$21$

w$22$

v$11$

v$21$

v$12$

v$22$

$($a$)$ Compute the forward propagation and generate the output. Use Relu for hidden layers

and Sigmoid activation function for output layer. $[2]$

$($b$)$ Compute the Softmax loss function for both outputs. $[1]$

$($c$)$ Let the initial weights that assumed be the weights $[$at time $($t$-1).$ Compute the weights

v$21,$ w$12$ and w$22$ at time t using SGD$.[1\mathrm{.}5]$

$($d$)$ Let the weight at time t be the ones computed in part $($c$).$ Compute the weights v$21,$

w$12$ and w$22$ at $($t $+1)$ when momentum is used.