Feature selection aims to select a subset of features that will be used in training. In general, there are three major types of feature selection strategy: filter method, wrapper method and embedded method.

a. Which type of feature selection strategy does Fisher Score belong to? Please justify your answer using 1-2 sentences.

b. Suppose we have six training examples shown below. Calculate Fisher scores for \(\theta_{0}\) and \(\theta_{1}\) and find out which one is more discriminative.

c. Which type of feature selection strategy does LASSO belong to? Please justify your answer using 1-2 sentences.

d. Denoting the vector of feature coefficients as \(\mathbf{w}\), suppose we have a tuning parameter \(\lambda\) for LASSO (i.e. the LASSO penalty term is \(\lambda\|\mathbf{w}\|_{1}\) ). If \(\lambda\) goes to infinity, what would happen to w? Why?

Transcribed Image Text:

example I 2 3 4 5 as 6 0 label 33 30 96 86 78 29 92 36 80 35 + + + + 90 32 +