(b) You are to fit a linear regression model on the following dataset with 3 training...

Transcribed Image Text:

(b) You are to fit a linear regression model on the following dataset with 3 training samples. Each sample has 3 features and 1 label. Suppose the initial parameters for linear regression are initialized as follows: wo = 2, W₁ = 1, W2 = 0, w3 = -1, compute the Least Mean Squared (LMS) error for this model and the updated model parameters after 1 iteration with learning rate 0.1. [10 pts] = X = min W (c) Regularization is a technique often used to improve the generalizability of models. L2 regularization adds squared magnitude of model parameters as penalty term to the loss function. The following equation shows LMS loss (objective) function with L2 regularization for linear regression: 1 1 3 -2] -1 0 2 0.5 2 1 m y = n T [(y₁ − w ¹ x ₁ ) ² + x Σ w² i=1 Show that the optimal solution w* for the above equation is (XTX + \I)-¹X¹y. Hint: Derive the above formula with respect to w [10 pts] (b) You are to fit a linear regression model on the following dataset with 3 training samples. Each sample has 3 features and 1 label. Suppose the initial parameters for linear regression are initialized as follows: wo = 2, W₁ = 1, W2 = 0, w3 = -1, compute the Least Mean Squared (LMS) error for this model and the updated model parameters after 1 iteration with learning rate 0.1. [10 pts] = X = min W (c) Regularization is a technique often used to improve the generalizability of models. L2 regularization adds squared magnitude of model parameters as penalty term to the loss function. The following equation shows LMS loss (objective) function with L2 regularization for linear regression: 1 1 3 -2] -1 0 2 0.5 2 1 m y = n T [(y₁ − w ¹ x ₁ ) ² + x Σ w² i=1 Show that the optimal solution w* for the above equation is (XTX + \I)-¹X¹y. Hint: Derive the above formula with respect to w [10 pts]