Question: Your friend is building a spam email classification system using logistic regression. She performed cross-validation to choose the shrinkage penalty constant from the grid {10^(-3)
Your friend is building a spam email classification system using logistic regression. She performed cross-validation to choose the shrinkage penalty constant from the grid {10^(-3), 10^(-2), ..., 10^2}. She then plotted the train, cross-validation, and test error against = {10^(-3), 10^(-2), ..., 10^2} , and saw that the test error had not yet plateaued and continued to decrease even around = 10^2.
Realizing that trying out hyperparameter values > 10^2 might help reduce the prediction error even further, she repeated the experiments but this time chose from the grid {10^(-3), 10^(-2), ..., 10^5, 10^6}, plotting the three types of errors (train, cross-validation, test) against again. She then chose = 10^4 as the optimal hyperparameter value that minimizes the cross-validation error, and reported error(Dtest; ) as the performance of the logistic regression model.
Is this procedure correct for reporting test error of logistic regression on the machine learning task? If not, mention and explain any precautions/corrections to the procedure your friend should have taken or been aware of during their experimentation.
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
