Problem $4\mathrm{.}3$ Using the non$-$negative homogeneity property of the ReLU function $($see problem $3\mathrm{.}5),$ show that:

ReLU B$,+1,-52,$ ReLU $(3.+10-820$x$|]=161,-$ ReLU $[\_1$

$[$A$01,$ B$,+52,$ ReLU $1$

$(4\mathrm{.}18)$

where $^,$ and $1,$ are non$-$negative scalars. From this, we see that the weight matrices can be rescaled by any magnitude as long as the biases are also adjusted, and the scale factors can be re$-$applied at the end of the network.