PLEASE DO NOT USE CHATGPT$/$AI$!($I$\text{'}$ll know if you do$)$

Consider a linear regression problem with weights. Specifically, given training data

with feature $x_1,$dots,$x_{N}in{R}^d$ and outcome $y_1,$dots,$y_N$inR suppose we want to find $win{R}^d$ that

minimizes

$L(w)$:$=\frac{1}{2}{\displaystyle \underset{i=1}{\overset{N}{}}}{r}_i(w^{TT}{x}_i-y_t{)}^2$

where $r_1,$dots,$r_N>0$ are positive weights. Note that we worked out the unweighted setting

for all $\{$:$i)$ in the class. In this problem, we will generalize some of those ideas to the

weighted setting where the weight $r_i$ can be different for each of the training examples. $($Hint:

check your answer when $r_i=1$ for all $i)$

$($a$)$ Show that $L(w)=(xw-Y{)}^{TT}R(xw-Y)$ for an appropriate definition of $x,Y$ and $R$

with respect to $x_i,y_i,r_i$ for $i=1,$dots,$N.$

$($b$)$ Recall that if all $r_i\text{'}$s are $1,$ we showed that $L(w^{*})=mi{n}_{w}L(w)$ when $w^{*}=(x^{TT}x{)}^{-1}{x}^{TT}y.$

By computing the derivative $gra{d}_{w}L(w)$ and setting that to zero, generalize the above result

to the weighted setting and give the new value of $w^{*}$ that minimizes $L(w)$ in closed form

as a function of $x,R,$ and $Y.$

$($c$)$ Suppose we have a training set $(x_i,y_i{)}_i=1,$dots,$N$ where $x_i$ are fixed and the distribution of $y_i$

is

In other words, $y_i$ has mean $w^{TT}{x}_i$ and variance ${}_i^2.$ Show that finding the maximum

likelihood estimate of $w$ reduces to solving a weighted linear regression problem. State

clearly what $r_i\text{'}$s are in terms of ${}_i\text{'}$s$.$