Recall that in lectures we showed that the Logistic Regression for binary classification boils down to solving the following optimization problem (training error) over n training samples: n T f(w) = Σlog (1 + e¯Yiw™; Xi -4:20-21) i=1 a) Compute the gradient of f(w). b) Please write the pseudocode for using GD to optimize the f(w). c) Argue that

 

Transcribed Image Text:

Recall that in lectures we showed that the Logistic Regression for binary classification boils down to solving the following optimization problem (training error) over n training samples: n T f(w) = Σlog (1 + e¯Yiw™; Xi -4:20-21) i=1 a) Compute the gradient of f(w). b) Please write the pseudocode for using GD to optimize the f(w). c) Argue that if data is linearly separable (i.e., can be perfectly classified by a linear model), in the solution obtained by GD, some of the coefficients converge to infinity.

Expert Answer

a) Compute the gradient of f(w). The gradient of f(w) is: df(w)/dw = (y - 1/ (1+ e^… View the full answer