Question: In some applications, a classifier is allowed to 'reject' a test example rather than classifying it. Consider a case in which the coast of misclassification
In some applications, a classifier is allowed to 'reject' a test example rather than classifying it. Consider a case in which the coast of misclassification is $10 but the cost of having a human manually make the decision is only $3. Suppose our problem is a two class problem, so either C=0 or C=1, and P(C=0|x) = 1 - P(C=1|x). So for our classifier, the possible actions are:
| a0 | Classify as class 0 (C=0) |
| a1 | Classify as class 1 (C=1) |
| a2 | Reject (send to human) |
and losses are:
| Loss ($) | Description | |
| 00 | 0 | Classify as class 0 when the true class is also 0 |
| 11 | 0 | Classify as class 1 when the true class is also 1 |
| 01 | 10 | Classify as class 0 when the true class is 1 |
| 10 | 10 | Classify as class 1 when the true class is 0 |
| 20 | 3 | Reject when the true class is 1 |
| 21 | 3 | Reject when the true class is 0 |
a. What is the expected risk for each action? (Meaning: write a formula for R(ai | x) in terms of P(C=1|x) for each of the three actions.)
b. What is the optimal decision rule? (Meaning : write something of the form: choose a0 if .... , chose a1 if ... otherwise reject.)
c. Suppose P(C=1|x) = 0.2 for some input x, what should the classifier's action be?
d. Suppose P(C=1|x) = 0.4 for some input x, what should the classifier's action be?
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
