The rectifier function r(2 ) = if x r(x) and s(x)

The rectifier function 0 if < 0 if c > 0 is used in artificial neural networks to model the firing of neurons. However, r@) is not differentiable at 0. Differentiability can improve the stability and performance of neural networks. Two common differentiable approximations to r@) are the softplus function p@) = log (1 + eX) and the swish function In this assignment, you may use without proof the facts that p@) > r@) and s@) r@) for all a; , and p@) and r@) are both continuous. 2. (a) Explain why p@) approximates r@) well for large (positive and negative) values of a;. (b) Explain why s@) approximates r@) well for large (positive and negative) values of a;.