[Statistical Learning] Consider building a model of the relationship between a response Y and the three predictors X_1, X_2, X_3 using the n independent observations (y_1, x_11, x_12, x_13), . . . ,(y_n, x_n1, x_n2, x_n3) and suppose that the observed predictor vectors xj = (x_1j ... x_nj)T are standardised so that in particular average x_j=i=1n x_ij/n =0; j=1,2,3.

(1) Show that the estimated slopes _1j in the simple linear regression models Y = _0j + _1j*X_j + can be written as

_1j =<x_j, y>/<x_j, x_j>; j=1,2,3,

where the vector y = (y_1 ... y_n)T and < , > is the inner or scalar product.

(2) Let Z1 and Z2 denote the first and second linear combinations of X1, X2, X3 in the partial least squares model of Y and suppose that the observed predictor vectors x1, x2, x3 are orthogonal.

(i) Show that the estimated slopes _1j in the simple linear regression models X_j = _0j + _1j*Z_1 + _j can be written as

_1j=<x_j, y>/ <z_1, z_1>; j=1,2,3,

where the vector z_1 = (z_11 ... z_n1)T contains the computed values for the first linear combination.

(ii) Show also that <z1, y> = <z1, z1>

(iii) Explain how you would construct the second linear combination Z_2.

(iv) Using the above results, or otherwise, show that the computed values for the second linear combination Z2 are all zeros, and hence partial least squares stops after m = 1 steps if the observed predictors are orthogonal.