Q$4.$ Machine Learning Evaluation $($ROC Curve$)($Total $12$pt$)$

You have been asked to develop a classification model for diagnosing whether a patient is infected with a certain disease. To help you construct the models, your collaborator has provided you with a small training set $\backslash ((\backslash$mathrm$\{$N$\}=10)\backslash )$ with equal number of positive and negative examples. You tried several approaches and found two most promising models, $\backslash (\backslash$mathrm$\{$C$\}\_\{1\}\backslash )$ and $\backslash (\backslash$mathrm$\{$C$\}\_\{2\}\backslash ).$

The outputs of the models in terms of predicting whether each of the training examples belong to the "positive $\backslash ((+)\backslash )"$ class are summarized in the table below. The first row shows the probability a training example belongs to the positive class according to classifier $\backslash (\backslash$mathrm$\{$C$\}\_\{1\}\backslash ),$ while the second row shows the same information for classifier C $2.$ The last row indicates the true class label of the $10$ training examples.

For each model, we will evaluate different thresholds within the range of $\backslash ([0,1]\backslash ),$ and a sample with probability $\backslash ($ P$\backslash$left$($y$=+\backslash$mid C$\_\{$x$\}\backslash$right$)\backslash )(\backslash ($ x $\backslash )$ is either $1$ or $2)$ that is lower than this threshold will be estimated as$,\backslash (-\backslash$quad $\backslash )$ or $+$ if greater than this threshold.

By varying the thresholds $($referred to the lecture slide on ROC$),$ you can study the model performance and draw the ROC.

$($a$)(8$pt$)$ Draw the corresponding ROC curves for both classifiers on the same plot.

$($b$)(4$pt$)$ Which classifier can be considered better? Why?