Homework #5 Constant predictions 6 3. (9 points) One model that is even simpler than the linear model we talked about in lecture is the constant model = a, i.e. we predict exactly the same y for every observation. We might do this if we had no predictor variables. Or, if our predictor variable were categorical (e.g., gender; or treatment vs. control group), we might make a different prediction for each gender, estimating a constant model within each group. One benefit of studying the constant model is that it is a simple context in which we can build our intuition for how different loss functions differ from each other. Assume that we observe y.....y, and we choose a to minimize the empirical risk of predicting a for every single y: R(a) == n L(y,a) (a) (2 points) If we use the L2 loss L(y,) = (-), show that the best-fitting estimate is the sample mean; i.e., = . (b) (3 points) If we use the Ll loss L(y,y) = y-y, show that the best-fitting estimate is the sample median. To simplify the problem, you may assume that n is odd, so the median is well-defined. Homework #5 7 (c) (2 points) Another option is to use what we might call the L0 loss L(y, y) = 1{y }. This loss is interesting because it doesn't care the least bit how far y is from , only whether y is perfectly predicted or not. This is a natural loss to use if y is a categorical feature with no ordinal structure. Find what is. You may assume that we have a data set where the loss-minimizing value of a is unique (i.e., assume away the possibility that you would have to break ties to estimate ). (d) (2 points) Finally, consider a weighted version of the L1 loss: |y- L(y, ) = if y> wy if y