Consider the linear regression model with deterministic regressor matrix X:

y = X + , i i.i.d. N (0, ) .

The error term is assumed to be strictly exogenous.

(a) Find the estimator e given by

~ =arg min { (y_i x_i ' )² + ' }

= arg min {( y- X)' (y X) + '}

where 0. Treat as given.

(b) Show that e is unique for > 0 even if the design matrix X does not have full column rank.

Hint: Examine your solution of (a). Recall that if a matrix D is symmetric and positive definite it is

invertible

(c) Assume X has full column rank. Derive an expression linking ~ to b, i.e. find a matrix Z such that ~ = Zb.

Hint: Z is the product of several matrices.

(d) Derive the bias and the covariance matrix of ~.

Hint: It might help to consider your solution found in (c).

(e) Recall that the Mean Squared Error (MSE) of an estimator ~ _k1 for the true parameter vector _Kx1

can be decomposed into

MSE(~) =E[(~ - )' (~ - )] = tr (var(~)) + Bias( ~)' Bias (~).

Briefly explain how the choice of affects the MSE and describe the underlying trade-off behind the

MSEs minimization