Question: A classifier is being tested on two datasets: Dataset 1 , with 1 0 0 positives and 1 0 0 negatives, and Dataset 2 ,

A classifier is being tested on two datasets: Dataset 1, with 100 positives and 100 negatives, and Dataset 2, with 100 positives and 500 negatives. The confusion matrices of the classifier on the two datasets are provided below. Dataset 1 Predicted + Predicted - Actual +8020 Actual -2080 Dataset 2 Predicted + Predicted - Actual +8020 Actual -100400 a) Calculate the Precision, Recall, TPR, and FPR for the classifier on Dataset 1 and Dataset 2. b) Based on your observations from these results, if you had to choose between the following two evaluation metric pairs: {precision, recall} and {TPR, FPR}, which one would you choose in the following scenarios and why? Provide brief explanations in context with your observations from the results above. The evaluation is required to be invariant to changes in the relative numbers of positives and negatives in the evaluation dataset. c) Compute the accuracy of the classifier on Dataset 2(you can leave your answer in fractions). Construct a trivial classifier that can achieve better accuracy on Dataset 2 without even looking at the attributes of the data. What is the accuracy of this trivial classifier?

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Programming Questions!