2. Cupcake Approval (k-NN classification). Fleur-de-Lis is a boutique bakery specializing in cupcakes. The bakers at Fleur-de-Lis like to experiment with different combinations of four major ingredients in its cupcakes and collect customer feedback; it has data on 150 combinations of ingredients with the corresponding customer reception for each combination classified as “thumbs up” (Class 1) or “thumbs down” (Class 0). To better anticipate the customer feedback of new recipes, Fleur-de-Lis has determined that a k-nearest neighbors classifier with k 510 seems to perform well.

Using a cutoff value of 0.5 and a validation set of 45 observations, Fleur-de-Lis constructs following confusion matrix and the ROC curve for the k-nearest neighbors classifier with k 510:

Predicted Feedback Actual Feedback Thumbs Up Thumbs Down Thumbs Up 13 1 Thumbs Down 1 30

As the confusion matrix shows, there is one observation that actually received thumbs down, but the k-nearest neighbors classifier predicts a thumbs up. Also, there is one observation that actually received a thumbs up, but the k-nearest neighbors classifier predicts a thumbs down. Specifically:

a. Explain how the probability of Thumbs Up was computed for Observation A and Observation B. Why was Observation A classified as Thumbs Up and Observation B was classified as Thumbs Down?

b. Compute the values of sensitivity and specificity corresponding to the confusion matrix created using the cutoff value of 0.5. Locate the point corresponding to these values of sensitivity and specificity on the Fleur-de-Lis’s ROC curve.

c. Based on what we know about Observation B, if the cutoff value is lowered to 0.2, what happens to the values of sensitivity and specificity? Explain. Use the ROC curve to estimate the values of sensitivity and specificity for a cutoff value of 0.2.

1.0 0.9 0.8- 0.7- 0.6- 0.5 0.4- Sensitivity 0.3- 0.2- 0.1- Random Classifier Optimum Classifier Fitted Classifier 0.0 T 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1 - Specificity