The figure below shows a multiple layer neural network deployed to predict the outcome of a numerical continuous target variable $y.$ The activation function in the first hidden layer is the sigmoid function, while the one in the second hidden layer is the Rectified Linear Unit $($ReLU$).$ The output layer uses an identity output function. The weight and bias parameters are shown along the edges in the network. Refer to part $($c$)$ for the red colored edge.

Input layer

Hidden layer $1$

Hidden layer $2$

Output layer

$($a$)$ Perform a forward propagation calculation through the network for the input observation vector $x=(-0\mathrm{.}5,0\mathrm{.}3).$

$($b$)$ The value of the target variable associated to the input vector $x=(-0\mathrm{.}5,0\mathrm{.}3)$ is $y=7.$ Calculate the loss associated with this training instance using an appropriate loss function.

$($c$)$ Let $w_2$ denote the weight corresponding to the red edge in the network. Compute the value of the gradient of the lossE$=\frac{1}{2}(y-o{)}^2$ with respect to this weight $w_2($note that $o$ denotes the output value of the network$).$

$($d$)$ Compute the update of the weight $w_2$ considering a learning rate of $0\mathrm{.}03$ and that the network has been trained with $L_2$ regularization with a penalty hyperparameter of $0\mathrm{.}002.$

$5$