Back$-$propagation Algorithm $2.$ Back$-$propagation Algorithm Once we set up the architecture of our $($feedforward$)$ neural network, our goal will be to find weight parameters that minimize our loss function. We will use the stochastic gradient descent algorithm $($which you learned in Lecture $4$ and revisited in lecture $5)$ to carry out the optimization. This involves computing the gradient of the loss function with respect to the weight parameters. Since the loss function is a long chain of compositions of activation functions with the weight parameters entering at different stages, we will break down the computation of the gradient into different pieces via the chain rule; this way of computing the gradient is called the backpropagation algorithm. In the following problems, we will explore the main step in the stochastic gradient descent algorithm for training the following simple neural network from the video:This simple neural network is made up of L hidden layers, but each layer consists of only one unit, and each unit has activation function f$.$ As usual, x is the input, z$\_$i is the weighted combination of the inputs to the i$^$t h hidden layer. In this one$-$dimensional case, weighted combination reduces to products:

z$\_1=$x w$\_1$

for i$=2...$ L: z$\_$i $=$f$\_$i$-1$ w$\_$i where f$\_$i$-$

We will use the following loss function:

$($y$,$ f$\_$L$)=($y$-$f$\_$L$)^2$

where y is the true value, and f$\_$L is the output of the neural network.and the true label y$.$ Let $\backslash$delta $\_$i$=/$ z$\_$i In this problem, we derive a recurrence relation between $\backslash$delta $\_$i and $\backslash$delta $\_$i$+1$ Assume that f is the hyperbolic tangent function:

f$($x$)=$tanh $($x$)$

f$^\text{'}($x$)=(1-$tanh $^2($x$))$

Which of the following option is the correct expression for $\backslash$delta $\_1$ in terms of $\backslash$delta $\_2?$

$\backslash$delta $\_1=(1-$f$\_1^2)$ w$\_2\backslash$delta $\_2$

$\backslash$delta $\_1=(1-$f$\_1^2)$ w$\_1\backslash$delta $\_2$

$\backslash$delta $\_1=(1-$f$\_2^2)$ w$\_2\backslash$delta $\_2$

$\backslash$delta $\_2=(1-$f$\_1^2)$ w$\_2\backslash$delta $\_1$