Regularized Least Squares

In this problem, you will analyze the solution to regularized least squares equation given as 11;}ng , Ax\": + IIMXIIS- Typically, regularization is used to solve the least squares problem with an ill-conditioned matrix A by using M : AI to "penalize" the 2norm of the solution. "Penalize" here means "introduce a penalty", or more straightfonvardly, to introduce an \"incentive" for the minimzation to keep the 2norm of x small. Regularization can also be used to reduce overtting of the solution by choosing M appropriately according to the distribution of the error in the data. By the normal equations, the solution to the above least squares problem is given as x : (ATA + MTM)'1ATb. Let A E Rm\" be a full rank matrix with m > n with the reduced SVD, A : 211:1 UiuiVlT! where ui E Rm,\" 6 R" are orthonormal vectors, and al- are the singular values. Let b E Rm be a vector. Part 1: (a) First show that when M = XI, the solution to the regularized least squares problem, X, is given as, n oju b X = 2 Vi. i= 1 (b) Let ) = 1. Show that