We consider the multiple linear regression model Y = X+e, (2) where Y = Rn, X...

Transcribed Image Text:

We consider the multiple linear regression model Y = X+e, (2) where Y = Rn, X = RnXP, and where the regression errors satisfy e~ N(0,I). The parameters of the model are R and > 0, with true values , . We assume that the first column of X represents the intercept, and X is of rank p. (a) Give the expressions of the least squares estimator of and the unbiased estimator 2 of 2. (b) Under the assumptions of the model what is the distribution of and ? Using these distributions, explain how to construct a 95% confidence interval for *.1, the first component of B. (c) For some 1 p < p, suppose that we partition X as X = (X1, X2), where X Rp1, and X2 = RnXp2, where p + p2 = p. We assume that X is of rank p. Suppose now that we remove all the predictors in X2 and fit the model (under the same error distribution assumption) Y = X + e. We partition the true regression coefficient as B. = (1+ * (3) i. Show that regardless of 42, the R of model (2) is typically larger than the R2 of model (3). We recall that the R of a linear regression model (with intercept) with fitted values is given by R = 1- - ||Y - || -1 (4 - 0)2 ii. Show that if +2 = 0, then Mallow's C will prefer model (3) to the full model (2). We recall that Mallow's C for a sub-model of model (2) with fitted values Y, and p columns is Cp = = ||Y - || 82 - (n - 2p1), where 2 is computed from the full model. We consider the multiple linear regression model Y = X+e, (2) where Y = Rn, X = RnXP, and where the regression errors satisfy e~ N(0,I). The parameters of the model are R and > 0, with true values , . We assume that the first column of X represents the intercept, and X is of rank p. (a) Give the expressions of the least squares estimator of and the unbiased estimator 2 of 2. (b) Under the assumptions of the model what is the distribution of and ? Using these distributions, explain how to construct a 95% confidence interval for *.1, the first component of B. (c) For some 1 p < p, suppose that we partition X as X = (X1, X2), where X Rp1, and X2 = RnXp2, where p + p2 = p. We assume that X is of rank p. Suppose now that we remove all the predictors in X2 and fit the model (under the same error distribution assumption) Y = X + e. We partition the true regression coefficient as B. = (1+ * (3) i. Show that regardless of 42, the R of model (2) is typically larger than the R2 of model (3). We recall that the R of a linear regression model (with intercept) with fitted values is given by R = 1- - ||Y - || -1 (4 - 0)2 ii. Show that if +2 = 0, then Mallow's C will prefer model (3) to the full model (2). We recall that Mallow's C for a sub-model of model (2) with fitted values Y, and p columns is Cp = = ||Y - || 82 - (n - 2p1), where 2 is computed from the full model.