Figure $11\mathrm{.}15$ Computational graph for batch normalization $($see problem $11\mathrm{.}5).$

Problem $11\mathrm{.}5*$ The forward pass for batch normalization given a batch of scalar values $\{z_i{\}}_i=1^{\text{I}}$

consists of the following operations $($figure $11\mathrm{.}15)$:

$f_1=E[z_i]$

$f_{2i}=x_i-f_1$

$f_{3i}=f_{2i}^2$

$f_4=E[f_{3i}]$

$f_5=\sqrt[\phantom{2}]{f_4+lon}$

$f_6=\frac{1}{f_5}$

$f_{7i}=f_{2i}{f}_6$

$z_i^{\text{'}}=f_{7i}+,$

where $E[z_i]=\frac{1}{\text{I}}{\displaystyle \underset{i}{\overset{?}{}}}{z}_i.$ Write Python code to implement the forward pass. Now derive the

algorithm for the backward pass. Work backward through the computational graph computing

the derivatives to generate a set of operations that computes del$\frac{z_i^{\text{'}}}{d}el{z}_i$ for every element in the

batch. Write Python code to implement the backward pass.