Question: Let x denote an n d matrix where rows are training points, y denotes an n 1 vector of corresponding output value, w denotes a

Let x denote an nd matrix where rows are training points, y denotes an n1 vector of corresponding output value, w denotes a d1 parameter vector and w*** denotes the optimal parameter vector. To make the analysis easier we will consider the special case where the training data is whitened (i.e.,xTTx=I ). For lasso regression, the optimal parameter vector is given by
w***=argminw12||y-xw||2+||w||1
where >0. a. Assume that wi*>0, what is the value of wi* in this case?b. Assume that wi*0, what is the value of wi* in this case ? c. From a and b what is the condition for wi* to be 0? How can you interpret that condition?d. Now consider ridge regression where the regularization term is replaced by (1/2)\lambda ||w||2. What is the condition for wi*=0? How does it differ from the condition you obtained in c?
 Let x denote an nd matrix where rows are training points,

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!