PLEASE DO NOT USE CHATGPT$/$AI$!($I$\text{'}$ll know if you do$)$

This problem compares ridge regression and another form of regularization for

linear regression, Lasso, which use $l_1-$norm as regularization.

Let $x$ denote an $nd$ matrix, where rows are training data, and $y$ denotes an $n1$ vector of

corresponding output values. In this problem we consider that the training data is whitened,

$x^{TT}x=I$

where I is the identity matrix. Given $>0,$ recall that ridge regression solves for

$w^{(2)}=argmi{n}_w\frac{1}{2}||y-xw|{|}_2^2+||w|{|}_2^2$

The lasso regression solves for

$w^{(1)}=argmi{n}_w\frac{1}{2}||y-xw|{|}_2^2+||w|{|}_1$

where $||w|{|}_1=|w_1|+|w_2|+$cdots$+|w_d|$ is the $l_1-$norm of $w=[w_1,w_2,$dots,$w_d{]}^{T}in{R}^d.$

$($a$)$ Show that under the whitened assumption both $\frac{1}{2}||y-xw|{|}_2^2+||w|{|}_2^2$ and $\frac{1}{2}||y-xw|{|}_2^2+$

$||w|{|}_1$ can be written as the following form:

$g(y)+{\displaystyle \underset{i=1}{\overset{d}{}}}f(x_i,y,w_i,)$

where $x_i$ is the $i-$th column of $x,g$ is a function of only $y,$ and $f$ is a function of

$x_i,y,w_i,.$ Then show that the optimal solutions $w^{(1)}$ and $w^{(2)}$ can be solved for each

feature separately.

$($b$)$ When does $w_i^{(2)}=0?$ Specify the condition using $y,x$ and $.$

$($c$)$ What does $w_i^{(1)}=0?$ Specify the condition using $y,x$ and $.$