Backpropagation $-$ Forward step $1$

Compute $4$ hidden neurons' output $(8$ orange boxes from left to right and from top to bottom

both before and after RELU, and the right most two network outputs in green$)$

Backpropagation $-$ Forward step $2$

Compute del$\frac{z}{d}$elw for all $12$ w$\text{'}$s $(12$ orange boxes from left to right and from top to bottom$)$

Backpropagation $-$ Backward step

Compute del$\frac{l}{d}$elz for all $4z^{\text{'}}\text{'}$s $(4$ orange boxes from left to right and from top to

bottom, also the two right most boxes for the error gradient in green$)$

Backpropagation $-$ final step

Compute del$\frac{l}{d}$elw for all hidden neuron's $w\text{'}$s $(8$ orange boxes from left to right and from top to bottom$)$