5.5 SVMs hands-on.

(a) Download and install the libsvm software library from:

http://www.csie.ntu.edu.tw/~cjlin/libsvm/.

(b) Download the satimage data set found at:

http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/.

Merge the training and validation sets into one. We will refer to the resulting set as the training set from now on. Normalize both the training and test vectors.

(c) Consider the binary classication that consists of distinguishing class 6 from the rest of the data points. Use SVMs combined with polynomial kernels

(see chapter 6) to solve this classication problem. To do so, randomly split the training data into ten equal-sized disjoint sets. For each value of the polynomial degree, d = 1; 2; 3; 4, plot the average cross-validation error plus or minus one standard deviation as a function of C (let the other parameters of polynomial kernels in libsvm, and

c, be equal to their default values 1).

Report the best value of the trade-o constant C measured on the validation set.

(d) Let (C; d) be the best pair found previously. Fix C to be C. Plot the ten-fold cross-validation training and test errors for the hypotheses obtained as a function of

d. Plot the average number of support vectors obtained as a function of d.

(e) How many of the support vectors lie on the margin hyperplanes?

(f) In the standard two-group classication, errors on positive or negative points are treated in the same manner. Suppose, however, that we wish to penalize an error on a negative point (false positive error) k > 0 times more than an error on a positive point. Give the dual optimization problem corresponding to SVMs modied in this way.
(g) Assume that k is an integer. Show how you can use libsvm without writing any additional code to nd the solution of the modied SVMs just described.
(h) Apply the modied SVMs to the classication task previously examined and compare with your previous SVMs results for k = 2; 4; 8; 16.