Projects

Small Project $1$: Linear Regression Models DUE DATE : June $7,\text{'}24$

In this project, you will get to use WEKA Tool.

Please see the References and Resources Section for guidelines on how to get this tool.

This assignment involves building and evaluating fault prediction models using Linear Regression, implemented in WEKA. Your task is to build models to predict the number of faults based on the other attributes of programs in the dataset. Each model is to be built and evaluated using $10-$fold cross validation on the fit data set, and then validated using the test data set.

The datasets have already been preprocessed for use in Weka.

You could download the datasets from the link under References & Resources.

Use the fit dataset to build models based on $10-$fold cross validation. When you build the model, you will get several statistical indicators, the measures of the quality of fit $($in the case of fit data$)$ and the predictive quality $($for the test data$),$ at the end of each run, as listed below:

Correlation coefficient

Mean absolute error $($also called AAE, which stands for Average Absolute Error$)$

Root mean squared error

Relative absolute error

Root relative squared error

The Linear regression models could be built with three different options for attribute selection in WEKA.

No Attribute Selection

M$5$ method

Greedy method

You have to use each attribute selection method for building the models. Consequently, you will have three different models. Compare the models, how many and which independent variables were selected? After building the models, evaluate their performance by supplying the test data set. Compare the quality of fit and predictive quality for each model built. Also compare the qualities of fit and predictive qualities among all the different models respectively. Your comparisons should not be based on just one parameter. Use all the statistical indicators $($mentioned hereabove$)$ provided by Weka to perform the comparisons.

Don't forget to include all the results based on the $10-$fold cross validation and the test data set for each model.