Regression

Consider we have the following training set, with inputs x R2 and labels y R. 21 22 y -1 2 4 0 -1 2 2 0 0 -1 -2 -4 0 1 -1 Our goal is to find a regression model that could regress r to our target value y. To do this, we are going to use a linear regression model. Namely, we are going to model our data by assuming the relationship y = w7x + wo + w" (x) + = [1] where (2) maps from R2 to homogeneous coordinates in R, and e~ N(0,0%) is 0-mean Gaussian noise. We call o a feature mapping of x and this feature mapping allows us to absorb the bias wo into the vector w, as we did previously for the perceptron and logistic regression. 1(a). With this feature mapping, we can write down the transformed data matrix as X = ((31) ... (en)] Using the formula we discussed in class, compute the closed form solution for w with the given training set. 1(b). Recall that the loss function for linear regression with feature mapping o is l(w) w" (1:) - w.)? Using your closed form solution obtained in the previous question, calculate the training loss. 1(c). Now suppose that our original dataset had been scaled such that one of the features was much larger than the other. Specifically, suppose that the second feature was scaled by a factor of 100 such that the dataset was 29y -1 2004 0 -100 2 2 0 0 -1 -200 -4 0 100 Using the expression for the closed form solution, show that this scaling does not affect the optimal training loss. What is the new value for the closed form solution w? 1(d). In part 1(e), you showed that scaling one of the features in linear regression does not affect the optimal training loss for plain linear regression. Would the same thing necessarily be true if ridge regression were used with some regularization parameter x > 07 Explain