Exercise $3$: Linear Regression

Let us consider a polynomial fitting problem. We assume the following model:

$y={}_0+{}_1{L}_1(x)+{}_2{L}_2(x)+$dots$+{}_p{L}_p(x)+lon,$

where $L_p(x)$ is the $p-$th Legendre polynomial, ${}_j$ are the coefficients, and $lon$ is the error term. In Python, if

you have specified a list of values of $x,$ evaluating the Legendre polynomial is quite straight forward:

import numpy as np

from scipy.special import eval$\_$legendre

$x=np.$linspace $(-1,1,50)$ # $50$ points in the interval $-1,1$

$L4=$ eval$\_$legendre $(4,x),$# evaluate the $4$ th order Legendre polynomial for $x$

$($a$)$ Let ${}_0=-0\mathrm{.}001,{}_1=0\mathrm{.}01,{}_2=0\mathrm{.}55,{}_3=1\mathrm{.}5,{}_4=1\mathrm{.}2,$ and let $lonGaussian(0,{0\mathrm{.}2}^2).$ Generate $50$

points of $y$ over the interval $x=np.$ linspace $(-1,1,50).$ That is$,$

$x=np.l$ inspace $(-1,1,50)$#$50$ points $in$ the interval $[-1,1]$

$y=$dots fill this line

Scatter plot the data.

$($b$)$ Given the $N=50$ data points, formulate the linear regression problem. Specifically, write down the

expression

widehat$()=argmi{n}_{}||y-x|{|}^2.$

What are $y,x,$ and $?$ Derive the optimal solution for this simple regression problem. Express your

answer in terms of $x$ and $y.$

$($c$)$ Write a Python code to compute the solution. Overlay your predicted curve with the scattered plot.

For solving the regression problem, you can call numpy.linalg.lstsq$.$

$($d$)$ For the $y$ you have generated, make $5$ outlier points using the code below:

#dots,

idx $=[10,16,23,37,45]\frac{?}{?}$# these are the locations $of$ the outliers

$y[idx]=5\frac{?}{?}$# set the outliers $to$ have a value $5$

#dots,