Part A

Suppose that we use some statistical learning method to make a prediction for the response Y for a particular value of the predictor X.

Carefully describe how we might estimate the standard deviation of

our prediction.

Part B

For parts (a) through (c), indicate which of i. through iv. is correct. Justify your answer.

(a) The lasso, relative to least squares, is:

i. More flexible and hence will give improved prediction accuracy when its increase in bias is less than its decrease in

variance.

ii. More flexible and hence will give improved prediction accuracy when its increase in variance is less than its decrease

in bias.

260 6. Linear Model Selection and Regularization

iii. Less flexible and hence will give improved prediction accuracy when its increase in bias is less than its decrease in

variance.

iv. Less flexible and hence will give improved prediction accuracy when its increase in variance is less than its decrease

in bias.

(b) Repeat (a) for ridge regression relative to least squares.

(c) Repeat (a) for non-linear methods relative to least squares.