[Designing the optimal predictor for continuous output spaces] We studied in class that the "Bayes Classifier" f := arg maxy P[Y |X] is optimal in the sense that it minimizes generalization error over the underlying distribution, that is, it maximizes Ex,y[1[g(x) = y]]. But what can we say when the output space Y is continuous? Consider predictors of the kind g : X ? R that predict a real-valued output for a given input x ? X . One intuitive way to define the quality of of such a predictor g is as

Q(g) := Ex,y[(g(x) ? y)^2].

Observe that one would want a predictor g with the lowest Q(g).

(i) Show that if one defines the predictor as f(x) := E[Y |X = x], then Q(f) ? Q(g) for any g, thereby showing that f is the optimal predictor with respect to Q for continuous output spaces.

(ii) If one instead defines quality as Q(g) := Ex,y|g(x) ? y|, which f is the optimal predictor? Justify your reasoning.

3 [Designing the optimal predictor for continuous output spaces] We studied in class that the "Bayes Classifier" (f := arg max, P[Y|X] ) is optimal in the sense that it minimizes generalization error over the underlying distribution, that is, it maximizes Ex,y [1 [g(x) = y]]. But what can we say when the output space ) is continuous? Consider predictors of the kind g : A' -> R that predict a real-valued output for a given input x E Y. One intuitive way to define the quality of of such a predictor g is as Q(g) := Ex,ul(g(x) - y)2]. Observe that one would want a predictor g with the lowest Q(g). (i) Show that if one defines the predictor as f(x) := E[Y|X = x], then Q(f)