$3\mathrm{.}1$ Part $1$: Classification:

This part is concerned with the file: $/$DataMining$/$data$/$arff$/$UCI$/$credit$-$g$.$arff.

The data was supplied by the Garavan Institute and J$.$ Ross Quinlan, NSW$,$ Australia. The main goal here is to achieve the highest classification accuracy with the lowest amount of over$-$fitting.

$1.$ Run the following classifiers, with the default parameters, on this data: ZeroR, OneR, J$48,$ IBK and construct a table of the training and cross$-$validation errors. You can get the training error by selecting $$Use training set$$ as the test option. What do you conclude from these results? Provide your explanation.

Run No Classifier Parameters Training Error Cross$-$validation Error Overfitting

$1$ ZeroR None $30\mathrm{.}0\%30\mathrm{.}0\%$ None

$.....$

$2.$ Using the J$48$ classifier, can you find a combination of the C and M parameter values that minimizes the amount of overfitting? Include the results of your best five runs, including the parameter values, in your table of results. What is your conclusion? Provide your explanation.

$3.$ Reset J$48$ parameters to their default values. What is the effect of lowering the number of examples in the training set? Provide your explanation. Include your runs in your table of results.

$4.$ Using the IBk classifier, can you find the value of k that minimizes the amount of over$-$ fitting? Provide your explanation. Include your runs in your table of results.

$5.$ Try two other classifiers. Aside from ZeroR, which classifiers are best and worst in terms of predictive accuracy? Include $5$ runs in your table of results. Provide your analysis on these results.

$6.$ Compare the accuracy of ZeroR, OneR and J$48.$ What do you conclude? Give your

explanation on these results. $7.$ What $$golden nuggets$$ did you find, if any?

$8.$Use an attribute selection algorithm to get a reduced attribute set. How does the accuracy on the reduced set compare with the accuracy on the full set? Provide your explanation. Report Length: Up to two pages, not including the table of runs.