11.3 Linear regression.

(a) What condition is required on the data X in order to guarantee that XX>

is invertible?

(b) Assume the problem is under-determined. Then, we can choose a solution w such that the equality X>w = X>(XX>)yXy (which can be shown to equal XyXy) holds. One particular choice that satises this equality is w = (XX>)yXy. However, this is not the unique solution. As a function of w, characterize all choices of w that satisfy X>w = XyXy (Hint: use the fact that XXyX = X).