$[$Linear Regression$](8$ pts$)$

Suppose we have a weight vector $w=[w_1,w_2{]}^T$ with input vectors $x_{n}in{R}^2$ and

$y_{n}in\{0,1\}(y=wx=w_1{x}_1+w_2{x}_2).$ Let us initialize all the weights to be $0.$ Also,

suppose we have $N=2$ examples in our dataset:

$\{$:$[-1,-1{]}^T,y_2=1).$ Work out on paper the process of training an $l_2$ regularized $(=1)$

linear regression model with batch gradient descent $($learning rate $=1)$ on the above

dataset for two epochs $($steps$).$

$($a$)(1$ pts$)$ What is the value of the loss function at the beginning?

$($b$)(4$ pts$)$ What is the final state of the trained weight vector after $2$ steps, and the

corresponding value of the loss function? $($Hint: derive the partial derivative of

the loss function with respect to weights, and calculate their values after each

step$)$

$L=\frac{1}{N}{}_i=1^N(y_i-w{x}_i{)}^2+||w|{|}^2$

$($c$)(3$ pts$)$ Derive the formula of the closed$-$form solution for ridge regression. $($Hint:

first write out the loss function in matrix form$).$