David Porter is the management accountant at Spruce Bank, where the data science department is leading an initiative to predict whether loans will default or repay. The default rate in the training set is 15%. After building a model on the training set that predicts whether a loan will default or repay, the data scientist applies it to the validation set of 400 observations to evaluate its performance.

Required

1. Help the data scientist complete the confusion matrixes below for different model thresholds as in Exhibits 11-19 and 11-20.

2. Assume that Spruce Bank has $1,000 to invest in each loan of the validation sample. If Spruce Bank does not invest in a loan, it keeps the money in a risk-free investment at 3% a year for 3 years (ignore the time value of money). If Spruce invests in a loan that eventually repays, it receives 10% a year for 3 years. If Spruce invests in a loan that eventually defaults, Spruce loses 65% of the amount of the loan. Fill in the payoff matrix below as in Exhibit 11-21. Which model threshold should David and the data scientist use?

Transcribed Image Text:

EXHIBIT 11-19 Actual Outcomes Confusion Matrix at Cutoff Value of 0.50 for Pruned Decision Tree Validation Sample Default Repay Total Predicted Outcomes Default (Do Not Invest in Loan) True Positives (A) 1 False Positives (Y) 0 1 Repay (Invest in Loan) False Negatives (Z) 1 True Negatives (B) 8 9 Total 2 8 10