Consider the logistic regression. We observe (X, Y), i = 1,. ..., n, where X Rd...

Transcribed Image Text:

Consider the logistic regression. We observe (X, Y), i = 1,. ..., n, where X Rd and Y {-1,1}. We draw X; from a normal distribution N(0, Idxd) and (1/d,...,1/d). Given X, and , we draw Y, from the following assume 3* = Bernoulli distribution P(Y = 1) = 1 P(Y = -1) = = 1 1+ exp(-XB*)" Consider using gradient descent to find the maximum likelihood estimator of this model. (a) Implement gradient descent method with the initial point 3(0) (0,...,0) and the backtracking line search. Please use R. or Python to implement it from the scratch and you need to submit your R or Python script. = (b) Draw plots of convergence rate when n = 2000, d = 5 and n = 5000, d = 5. In each plot, the x-axis is the number of step k and y-axis is log || 3(k) - 3*||. Consider the logistic regression. We observe (X, Y), i = 1,. ..., n, where X Rd and Y {-1,1}. We draw X; from a normal distribution N(0, Idxd) and (1/d,...,1/d). Given X, and , we draw Y, from the following assume 3* = Bernoulli distribution P(Y = 1) = 1 P(Y = -1) = = 1 1+ exp(-XB*)" Consider using gradient descent to find the maximum likelihood estimator of this model. (a) Implement gradient descent method with the initial point 3(0) (0,...,0) and the backtracking line search. Please use R. or Python to implement it from the scratch and you need to submit your R or Python script. = (b) Draw plots of convergence rate when n = 2000, d = 5 and n = 5000, d = 5. In each plot, the x-axis is the number of step k and y-axis is log || 3(k) - 3*||.

Answer rating: 100% (QA)

The gradient descent method is an optimization algorithm used to fin... View the full answer