Suppose we have p predictors and do forward selection to obtain a sequence of linear regression models f0, f1, . . . fp trained on our training set, where f1 has 1 non-zero coefficient (1 predictor in the model) and generally fj has up to j non-zero coefficients (0 j p), corresponding to j predictors in the model. Depending on the context I'll use fj to refer to both the (best) subset of the j predictors found via forward selection, and the linear regression model fitted on those predictors.

Similarly, we do backwards selection to get a sequence of models bp, bp1, . . . , b0 (bi has i predictors), and best subset selection for the sequence h0, . . . , hp. (best subset selection is computationally infeasible in general, but you may assume that we are able to examine all subsets of size k and use the best subset to train hk.)

(a) (2 points) Compare the predictors in and accuracy of f1 and h1.

(b) (2 points) Compare the predictors in and accuracy of b1 and h1.

(c) (2 points) Is there a size i, for (0 < i < p), where we are guaranteed bi is identical to hi?

(d) (2 points) Are the predictors in hj a subset of the predictors in hj+1?