Average error of least squares. In this problem, we consider whether a linear model is accurate, on average, for certain groups of examples. Given a data set D = {(x1, y1), . . . ,(xn, yn)} with xi R d , yi R for i = 1, . . . , n, define the ith prediction yi(w) made by a linear model with parameters w to be yi(w) = xi^T w. We say that a model with parameters w is accurate, on average, for the set of examples S {1, . . . , n} if X iS (yi yi(w)) = 0.

(a) Let's suppose that we are fitting a model with an offset: (xi)d, the last entry of xi , is equal to 1, for each i = 1, . . . , n. We will compute w using least squares, by solving minimize Xn i=1 (yi xi^T w) 2 with variable w R d . Show that the resulting linear model is accurate, on average, for the full set of examples S = {1, . . . , n}.