Question: 4.5.1 When Is the Lasso Solution Unique? The solution bbl of the minimization problem (4.4) is not always unique. In this exercise, we prove that
4.5.1 When Is the Lasso Solution Unique?
The solution bbl of the minimization problem (4.4) is not always unique. In this exercise, we prove that the fitted value bfl = Xbbl is always unique, and we give a criterion that enables us to check whether a solution is unique or not.
1. Let bb(1)
l and bb(2)
l be two solutions of (4.4) and set bb =
bb(1)
l + bb(2)
l
=2. From the strong convexity of x!kxk2 prove that if Xbb(1)
l 6= Xbb(2)
l , then we have kY ????Xbbk2+ljbbj1 <
1 2
kY ????Xbb(1)k2+ljbb(1)j1+kY ????Xbb(2)k2+ljbb(2)j1
:
Conclude that Xbb(1)
l = Xbb(2)
l , so the fitted value bfl is unique.
2. Let again bb(1)
l and bb(2)
l be two solutions of (4.4) with l > 0. From the optimality Condition (4.2), there exists bz(1) and bz(2), such that
????2XT (Y ????Xbb(1)
l )+lbz(1) = 0 and ????2XT (Y ????Xbb(2)
l )+lbz(2) = 0:
Check that bz(1) =bz(2). We write henceforth bz for this common value.
3. Set J =
j : jbz jj = 1
. Prove that any solution bbl to (4.4) fulfills
[bbl ]Jc = 0 and XTJ XJ [bbl ]J = XTJ Y ????
l 2
bzJ :
4. Conclude that
”when XTJ XJ is nonsingular, the solution to (4.4) is unique.”
In practice we can check the uniqueness of the Lasso solution by first computing a solution bbl , then computing J = n j : jXTj (Y ????Xbbl )j = l=2 o , and finally checking that XTJ XJ is nonsingular.
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts