Problem $1-$ Numerical Differentiation $(50$ Points$)$

As we will discuss later in the course, an important feature of machine learning $($ML$)$ models is that they

employ non$-$linear transformations, often referred to as the $$activation function$,$ which mimics the biology

of how our own neurons process information. One such example is tan$1($x$).$ Having an intuition for both

this function and its derivative will be imperative towards grasping how ML models learn.

Consider y$=$ tan$1($x$)$

$($A$)$ Describe the behavior of this function by describing $(1)$ why it is considered non$-$linear and $(2)$ what

happens to very large positive and negative inputs. $($max $2$ sentences$)$

$($B$)$ Search up online $($or compute by hand if you really want to$...)$ the derivative of y $=$ tan$1($x$).$ How

does the derivative explain the behavior of this function with very large inputs? This is the inherent

$$regularizing$$ property of arctan$.($max $2$ sentences$)$

$($C$)$ What is a numerical method, and how does it differ from analytical methods? Give an example

where a numerical method may be preferred to an analytical one. $($max $3$ sentences$)$

$($D$)$ Compute $$y

$$x x$=1$ analytically.

$($E$)$ Compute $$y

$$x x$=1$ numerically using a forward difference with h$=0\mathrm{.}01.$

$($F$)$ Compute $$y

$$x x$=1$ numerically using a second$-$order Taylor expansion about x$=0\mathrm{.}75.$

$($G$)$ What is the error $$ in the numerical approximation in $2$E$/2$F$?$

$($H$)$ To the nearest integer fraction of $1($e$.$g$.,1$

$1$

$5,$

$100,$ etc.$),$ what is the largest hyou can choose to achieve

$<0\mathrm{.}0001$ when using a forward difference? Note: his commonly referred to in the engineering field

as the $$sampling rate.$$