In this sub$-$question, we discuss the common evaluation metrics for imbalanced dataset. Suppose we have a

validation dataset and for some $\backslash$rho in $(0,1),$ we assume that $\backslash$rho fraction of the validation examples are positive

examples $($with label $1),$ and $1\backslash$rho fraction of them are negative examples $($with label $0).$

Define the accuracy as

A$$ #examples that are predicted correctly by the classifier

#examples

$($i$)[1$ point $($Written$)]$ Show that for any dataset with $\backslash$rho fraction of positive examples and $1\backslash$rho fraction of

negative examples, there exists a $($trivial$)$ classifier with accuracy at least $1\backslash$rho $.$

The statement above suggests that the accuracy is not an ideal evaluation metric when $\backslash$rho is close to $0.$ E$.$g$.,$

imagine that for spam detection $\backslash$rho can be smaller than $1\%.$ The statement suggests there is a trivial classifier

that gets more than $99\%$ accuracy. This could be misleading $99\%$ seems to be almost perfect, but actually

you don$$t need to learn anything from the dataset to achieve it$.$

Therefore, for imbalanced dataset, we need more informative evaluation metrics. We define the number of true

positive, true negative, false positive, false negative examples as

TP $$#positive examples with a correct $($positive$)$ prediction

TN $$#negative examples with a correct $($negative$)$ prediction

FP $$#negative examples with a incorrect $($positive$)$ prediction

FN $$#positive examples with a incorrect $($negative$)$ prediction

Define the accuracy of positive examples as

A$1$ TP

TP $+$FN $=$ #positive examples with a correct $($positive$)$ prediction

#positive examples

Define the accuracy of negative examples as

A$0$ TN

TN $+$FP $=$ #negative examples with a correct $($negative$)$ prediction

#negative examples

We define the balanced accuracy as

A$1$

$2($A$0+$A$1)$

With these notations, we can verify that the accuracy is equal to A $=$

TP$+$TN

$(4)$

TP$+$TN$+$FP$+$FN$.$