Show that the gradient of a loss function $L$(\backslash$vu$,\backslash$gamma$,\backslash$beta$)$$ with respect to $$\backslash$vu$ can be written in the form $

abla$\_\backslash$vu L$=$s$\backslash$mW$^\backslash$perp

abla$\_\backslash$vw L$ for some $s$$,$ where $$\backslash$mW$^\backslash$perp$=\backslash$left$(\backslash$mI$-\backslash$frac$\{\backslash$vu$\backslash$vu$^\backslash$top$\}\{||\backslash$vu$||^2\}\backslash$right$)$$$.$

Note that $\backslash$footnote$\{$As a side note: $$\backslash$mW$^\backslash$perp$ is an orthogonal complement that projects the gradient away from the direction of $$\backslash$vw$$,$ which is usually $($empirically$)$ close to a dominant eigenvector of the covariance of the gradient. This helps to condition the landscape of the objective that we want to optimize.$\}$ $$\backslash$mW$^\backslash$perp$\backslash$vu$=\backslash$mathbf$\{0\}$$$.$