Dim Reduction

Exercise $1$: Understanding Covariance

Objective: To understand how covariance indicates the relationship between dimensions in a dataset.

Task:

Consider a small dataset:

$x$:$1,2,3,4$

Y: $2,4,6,8$

Calculate the mean of $x$ and $Y.$

Compute the covariance between $x$ and $Y.$ Use the formula:

Cov$(x,Y)=(-x^{-})\frac{Yi-Y^{-}}{n}$

Discuss what the resulting covariance value tells you about the relationship between $x$ and $Y.$

Exercise $2$: Basic Concept of PCA

Objective: To conceptually understand the process and effects of PCA.

Task:

Imagine you have a dataset with two variables, $x$ and $Y,$ where $Y$ is perfectly correlated with $x.$ Describe what would happen if PCA were applied to this dataset.

Sketch a simple graph with a few points showing $x$ versus $Y,$ draw the principal component.

Explain in your own words how PCA reduces the dimensionality of this dataset and what information might be lost or retained.

Exercise $3$: Interpreting PCA Results

Objective: To learn how to interpret the results of PCA in a practical scenario.

Task:

Consider this outcome from a PCA on a dataset containing measurements of flowers: the first principal component explains $70\%$ of the variance, and the second principal component explains $20\%.$

Write a short interpretation of what these percentages tell you about the dataset.

Suppose we decide to keep only the first principal component:

Discuss what type of information about the flowers might be well represented by this component.

Consider what information might be lost by ignoring the second principal component.