Consider a data fitting problem, with first basis function 1(x)=1, and data set x(1),...,x(N),y(1),...,y(N). Assume the matrix A=[1xd] , has linearly independent columns and let ^ denote the parameter values that minimize the mean square prediction error over the data set. Let the N-vector r^d denote the prediction errors using the optimal model parameter ^. Show that avg(r^d)=0.

In other words: With the least squares fit, the mean of the prediction of errors over the data set is zero.

\textit{Hint}. Use the orthogonality principle (Az)r^, with z=e1.