$(7$ points$)$ In Lecture $12,$ we viewed both the simple linear regression model and the

multiple linear regression model through the lens of linear algebra. The key geometric

insight was that if we train a model on some design matrix X and true response vector

Y$,$ our predicted response Y$=$ X$$ is the vector in span$($X$)$ that is closest to Y$.$

In the simple linear regression case, our optimal vector is $$ $=[$$0,$

$$$1],$ and our design

matrix is

X $=$

$$

$$

$$

$$

$$

$1$ x$1$

$1$ x$2$

$.$

$.$

$.$

$.$

$.$

$.$

$1$ xn

$$

$$

$$

$$

$$

$=$

$$

$$

$||$

$1$n X:$,1$

$||$

$$

$$

This means we can write our predicted response vector as Y$=$ X

$$$0$

$$$1$

$=$$01$n $+$$1$X:$,1.$

In this problem, $1$n is the n$-$vector of all $1$s and X:$,1$ refers to the n$-$length vector

$[$x$1,$ x$2,...,$ xn$]$

$.$ Note, X:$,1$ is a feature, not an observation.

For this problem, assume we are working with the simple linear regression model,

though the properties we establish here hold for any linear regression model that contains

an intercept term.

$($a$)(3$ points$)$ Explain why Pn

i$=1$

ei $=0$ using a geometric property. $($Hint: $$e $=$ Y $$ Y$,$

and $$e $=[$e$1,$ e$2,...,$ en$]$

$.$ Think about how orthogonality applies here.$)$