data mining -irreducible data example

(a)[2 pts] Discuss the cases when PCA will fail.

(b)[3 pts] How do we quantify that it fails?

(c)[5 pts] Provide a minimal example of a dataset (specify the points as vec- tors of numbers) in which PCA will not work well for dimensionality reduction. Explain why. Hint: Think of 2D points and reduction to 1D.

(d)[10 pts] When linear PCA does not work we naturally go to kernels. For your example above, propose a suitable kernel (x) that you expect would do a better job if we do Kernel-PCA. Justify your selection. Compute the kernel matrix K(xi, xj ) = (xi)(xj ) for the first 3 points of your example data above.