In this problem, we are going to use simulated datasets to better understand how the square

of bias, variance, irreducible error, and MSE vary with model flexibility.

$($a$)4pts$ Generate a simulated dataset as follows:

def f$($x$)$:

return x $**5-2**$ x$*3$

def get$\_$sim$\_$data$($f$,$ sample$\_$size$=100,$ std$=0\mathrm{.}01)$:

x $=$ np$.$random.uniform$(0,1,$ sample$\_$size$)$

y $=$ f$($x$)+$ np$.$random.normal$(0,$ std$,$ sample$\_$size$)$

df $=$ pd$.$DataFrame$(\{\text{'}$x$\text{'}$: x$,\text{'}$y$\text{'}$: y$\})$

return df

In this dataset, what is the number of observations $n$ and what is the number of features

$p($different powers of $x$ are counted as different features$)?$ Write out the model used

to generate the data in equation form.

$($b$)[4$pts$]$ Fit the polynomial functions of degree from $0$ to $15$ using the simulated data in

$($a$)$:

$f_0(x)={}_0+$

$f_1(x)={}_0+{}_{1}x+$

$f_2(x)={}_0+{}_{1}x+{}_2{x}^2+$

vdots

$f_{15}(x)={}_0+{}_{1}x+{}_2{x}^2+{}_3{x}^3$cdots$+{}_{15}{x}^{15}+$

$($Hint: You may find

from sklearn. preprocessing import PolynomialFeatures

from sklearn.linear$\_$model import LinearRegression

useful.$)$

$($c$)[4$pts$]$ Predict the response at $x_0=0\mathrm{.}18$ using the fitted functions in $($b$).$

$($d$)4pts$ Repeat $(a)-(c)$ for $250$ times.

$($e$)4pts$ Use $($d$)$ to calculate the square of bias for the fitted polynomials hat$(f{)}_0(x_0),$hat$(f{)}_1(x_0),$cdots,hat$(f{)}_{15}(x_0).$

In this problem, we are going to use simulated datasets to better understand how the square

of bias, variance, irreducible error, and MSE vary with model flexibility.

$($a$)4pts$ Generate a simulated dataset as follows:

def f$($x$)$:

return x $**5-2**$ x$*3$

def get$\_$sim$\_$data$($f$,$ sample$\_$size$=100,$ std$=0\mathrm{.}01)$:

x $=$ np$.$random.uniform$(0,1,$ sample$\_$size$)$

y $=$ f$($x$)+$ np$.$random.normal$(0,$ std$,$ sample$\_$size$)$

df $=$ pd$.$DataFrame$(\{\text{'}$x$\text{'}$: x$,\text{'}$y$\text{'}$: y$\})$

return df

In this dataset, what is the number of observations $n$ and what is the number of features

$p($different powers of $x$ are counted as different features$)?$ Write out the model used

to generate the data in equation form.

$($b$)[4$pts$]$ Fit the polynomial functions of degree from $0$ to $15$ using the simulated data in

$($a$)$:

$f_0(x)={}_0+$

$f_1(x)={}_0+{}_{1}x+$

$f_2(x)={}_0+{}_{1}x+{}_2{x}^2+$

vdots

$f_{15}(x)={}_0+{}_{1}x+{}_2{x}^2+{}_3{x}^3$cdots$+{}_{15}{x}^{15}+$

$($Hint: You may find

from sklearn. preprocessing import PolynomialFeatures

from sklearn.linear$\_$model import LinearRegression

useful.$)$

$($c$)[4$pts$]$ Predict the response at $x_0=0\mathrm{.}18$ using the fitted functions in $($b$).$

$($d$)4pts$ Repeat $(a)-(c)$ for $250$ times.

$($e$)4pts$ Use $($d$)$ to calculate the square of bias for the fitted polynomials hat$(f{)}_0(x_0),$hat$(f{)}_1(x_0),$cdots,hat$(f{)}_{15}(x_0).$

$($f$)4pts$ Use $($d$)$ to calculate the variance for the fitted polynomials hat$(f{)}_0(x_0),$hat$(f{)}_1(x_0),$cdots,hat$(f{)}_{15}(x_0).$

$($g$)4pts$ Calculate the irreducible error based on the data generating process.

$($h$)4pts$ Calculate the MSE based on $($e$),($f$),$ and $($g$).$

$($i$)[6$pts$]$ Plot how the square of bias, variance, irreducible error, and MSE vary with the

degree of polynomials. Explain your findings.

$($f$)4pts$ Use $($d$)$ to calculate the variance for the fitted polynomials hat$(f{)}_0(x_0),$hat$(f{)}_1(x_0),$cdots,hat$(f{)}_{15}(x_0).$

$($g$)4pts$ Calculate the irreducible error based on the data generating process.

$($h$)4pts$ Calculate the MSE based on $($e$),($f$),$ and $($g$).$

$($i$)[6$pts$]$ Plot how the square of bias, variance, irreducible error, and MSE vary with the

degree of polynomials. Explain your findings.