Introduction to Business Analytics: Applied Linear Algebra and Python Coding.

Least squares classification with regularization. The file lsq_classifier_data.ipynb contains fea- ture n-vectors I1, ..., IN, and the associated binary labels, 41, ... , YN, each of which is either +1 or -1. The feature vectors are stored as an n x N matrix X with columns x1, ..., IN, and the labels are stored as an N-vector y. We will evaluate the error rate on the (training) data X, y and (to check if the model generalizes) a test set Xtest, Ytest, also given in lsq-classifier_data.ipynb. (a) (10 points) Least squares classifier. Find B, v that N minimize (2TB+v Y:)2 i=1 on the training set. Our predictions are then f(x) = sign(x3 +v). Report the classification error on the training and test sets, the fraction of examples where f(xi) + yi. There is no need to report the B, v values. (b) (10 points) Regularized least squares classifier. Now we add regularization to improve the gen- eralization ability of the classifier. Find B, v that N minimize (27B+v Y)2 + 1||$||2, i=1 where > 0 is the regularization parameter, for a range of values of 1. Please use the following values for 1: 10-1, 10, 107, 102, 10'. Suggest a reasonable choice of land report the corre- sponding classification error on the training and test sets. Again, there is no need to report the B, v values. Hint: plot the training and test set errors against log 10(a). import numpy as np # Dimensions and true w, v. np.random.seed(11) n = 50 N = 300 w_true = np.random.normal(size = n) v_true = 5 # Generate training data X, y. X = np.random.normal(size = (n, N)) y = np.sign(np.matmul(X.T,w_true) + v_true + 10*np.random.normal(size = N)) # Generate test data X_test, y_test. N_test = 100 X_test = np.random.normal(size = (n, N_test)) y_test = np.sign (np.matmul(X_test. T,w_true) + v_true + 10*np.random.normal(size = N_test)) Least squares classification with regularization. The file lsq_classifier_data.ipynb contains fea- ture n-vectors I1, ..., IN, and the associated binary labels, 41, ... , YN, each of which is either +1 or -1. The feature vectors are stored as an n x N matrix X with columns x1, ..., IN, and the labels are stored as an N-vector y. We will evaluate the error rate on the (training) data X, y and (to check if the model generalizes) a test set Xtest, Ytest, also given in lsq-classifier_data.ipynb. (a) (10 points) Least squares classifier. Find B, v that N minimize (2TB+v Y:)2 i=1 on the training set. Our predictions are then f(x) = sign(x3 +v). Report the classification error on the training and test sets, the fraction of examples where f(xi) + yi. There is no need to report the B, v values. (b) (10 points) Regularized least squares classifier. Now we add regularization to improve the gen- eralization ability of the classifier. Find B, v that N minimize (27B+v Y)2 + 1||$||2, i=1 where > 0 is the regularization parameter, for a range of values of 1. Please use the following values for 1: 10-1, 10, 107, 102, 10'. Suggest a reasonable choice of land report the corre- sponding classification error on the training and test sets. Again, there is no need to report the B, v values. Hint: plot the training and test set errors against log 10(a). import numpy as np # Dimensions and true w, v. np.random.seed(11) n = 50 N = 300 w_true = np.random.normal(size = n) v_true = 5 # Generate training data X, y. X = np.random.normal(size = (n, N)) y = np.sign(np.matmul(X.T,w_true) + v_true + 10*np.random.normal(size = N)) # Generate test data X_test, y_test. N_test = 100 X_test = np.random.normal(size = (n, N_test)) y_test = np.sign (np.matmul(X_test. T,w_true) + v_true + 10*np.random.normal(size = N_test))