Consider a fully connected neural network that has an input layer, one hidden layer and an output layer. The input layer has inputs $x_1$ and $x_2,$ while the output layer has one neuron that uses the Softmax activation function and produces two outputs $y_1^{(2)}$ and $y_2^{(2)}.$ The hidden layer has $3$ neurons and all of them use the ReLu activation function. The different notations used in the neural network are shown below:

$z_1=w_{11}{x}_1+w_{21}{x}_2,z_2=w_{12}{x}_1+w_{{22}^2}{x}_2,z_3=w_{13}{x}_1+w_{23}{x}_2,$

$y_1^{(1)}=$ReLu$(z_1),y_2^{(1)}=$ReLu$(z_2),y_3^{(1)}=$ReLu$(z_3)$

$s_1=y_1^{(1)}+y_2^{(1)},s_2=y_2^{(1)}+y_3^{(1)},$

$y_1(2)=$exp$\frac{s_1}{\{\begin{array}{ccc}\text{e}\text{x}\text{p}& (s_1)+\text{e}\text{x}\text{p}& (s_2)\\ y_2^{(2)}=\text{e}\text{x}\text{p}& \frac{s_2}{\text{e}\text{x}\text{p}(s_1)+\text{e}\text{x}\text{p}(s_2)}\end{array}}$

Suppose that $x_1=3,x_2=5,w_{11}=-10,w_{21}=7,w_{12}=2,w_{22}=5,w_{13}=4,w_{23}=-4.$ Compute the following gradients del$\frac{z_2}{d}el{x}_1,$del$\frac{z_1}{d}el{w}_{21},$ and del$\frac{s_1}{del{y}_1(1)}$