Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) have different purposes. The former operates on the
Question:
Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) have different purposes. The former operates on the total (unlabeled) dataset to find the directions that contain the maximum variance; the latter operates on labeled data to find the directions which are best at distinguishing the labeled classes. In general, the results (principal components and canonicals, respectively) will be quite different. However, for special examples of data the principal components and canonicals could be in the same directions. (1) Draw an example of two-class, two-dimensional data such that PCA and LDA find the same directions. (2) Draw an example of two-class, two-dimensional data such that PCA and LDA find totally different directions.