(Q4) Let X = (X X) be an n p random matrix such that Var((X);)...

Transcribed Image Text:

(Q4) Let X = (X₁ X₂) be an n × p random matrix such that Var((X¹);) = Σ, Vi, i.e. the Σ is the covariance matrix for row i of X (the ith column of XT). Assume that is a positive definite matrix with normed eigenvalue decomposition Σ = WAWI i) (10 points) Let Y; W(XT); be the vector of p scores for the i-th row of X. Show that the PCA representation preserves distance between the two vectors (XT); and (XT);, Vi, j € {1,..., n}, i.e. that i = ||XT — (x)} || = ||Y₁ – Yj|| - (1) - where ||u – v|| = (u − v)¹(u − v). [Hint: Use the properties of the various pieces of the eigenvalue decomposition.] ii) (10 points) Using the properties of traces of products of matrices and the defi- nition of E, show that: tr(Σ) = tr(A) showing that the sum of the eigenvalues is equal to the sum of the marginal variances. iii) (10 points) Suppose the random variables X₁, X₂ and X3 have the covariance matrix Σ = -2 0 5 0 00 2 1 -2 The eigenvalue-eigenvector pairs are shown for this covariance matrix in class. a) Calculate the variances of each of the principal components by calculating the variance of the linear combinations. Also calculate the covariance of all possible pairs of the principal components using the linear combinations. b) Calculate the proportion of total variance accounted for by the first, second and the third principal components, respectively. c) Verify the result shown in part (ii), i.e., sum of the eigenvalues is equal to the sum of the marginal variances. (Q4) Let X = (X₁ X₂) be an n × p random matrix such that Var((X¹);) = Σ, Vi, i.e. the Σ is the covariance matrix for row i of X (the ith column of XT). Assume that is a positive definite matrix with normed eigenvalue decomposition Σ = WAWI i) (10 points) Let Y; W(XT); be the vector of p scores for the i-th row of X. Show that the PCA representation preserves distance between the two vectors (XT); and (XT);, Vi, j € {1,..., n}, i.e. that i = ||XT — (x)} || = ||Y₁ – Yj|| - (1) - where ||u – v|| = (u − v)¹(u − v). [Hint: Use the properties of the various pieces of the eigenvalue decomposition.] ii) (10 points) Using the properties of traces of products of matrices and the defi- nition of E, show that: tr(Σ) = tr(A) showing that the sum of the eigenvalues is equal to the sum of the marginal variances. iii) (10 points) Suppose the random variables X₁, X₂ and X3 have the covariance matrix Σ = -2 0 5 0 00 2 1 -2 The eigenvalue-eigenvector pairs are shown for this covariance matrix in class. a) Calculate the variances of each of the principal components by calculating the variance of the linear combinations. Also calculate the covariance of all possible pairs of the principal components using the linear combinations. b) Calculate the proportion of total variance accounted for by the first, second and the third principal components, respectively. c) Verify the result shown in part (ii), i.e., sum of the eigenvalues is equal to the sum of the marginal variances.

Answer rating: 100% (QA)

Q4 i Let Yi WXi and Yj WXj be the vectors of principal component scores ... View the full answer