(e) (10 points) Now assuming the data are one-dimensional, the training dataset consists of two

samples x1 = 0.5 and x2 = 1, and the test sample x = 0.75. The true parameter ??

0 = 0, ??

1 = 1,

the noise variance is given by ?2

1 = 1, ?2

2 = 1. Plot the MSE (Bias square plus variance) as a

function of the regularization parameter ?.

(f) (5 points) Now change the test sample to be a x = 0.1, and keep everything else to be same as in

the previous question. Plot the MSE (Bias square plus variance) as a function of the regularization

parameter ?, and comment on the difference from the previous result.

3. Locally weighted linear regression and bias-variance tradeoff. (40 points) COHSider a dataSEt With 71 data 130111135 (957213103 951 6 RP, following the following linear model yi:.3*T'Iz-+q. i:l,....n, where Si N N(0, 0'12) are independent (but not identically distributed) Gaussian noise with zero mean and variance 0?. (a) (5 points) Show that the ridge regression which introduces a squared 122 norm penalty on the parameter in the maximum likelihood estimate of B can be written as follows A 500 : exam/gnaw Mir/(Xe y) + AIIBHE} or property dened diagonal matrix l/V, matrix X and vector y. 10 points) Find the closeform solution for EM) and its distribution conditioning on {931} l (c) 5 points) Derive the bias as a function of A and some xed test point 1:. ( j 5 points) Derive the variance term as a function of A. l 10 points) Now assuming the data are onedimensional, the training dataset consists of two samples 331 = 0.5 and 272 = 1, and the test sample a: = 0.75. The true parameter 55' = 0, Bf = 1, 'he noise variance is given by 0% : 1, 0% : 1. Plot the MSE (Bias square plus variance) as a unction of the regularization parameter A. (f ) 5 points) Now change the test sample to be a a: : 0.1, and keep everything else to be same as in 'he previous question. Plot the MSE (Bias square plus variancie) as a function of the regularization parameter A, and comment on the difference from the previous result