Problem $1.[30$ points$]$ Consider a binary training data S $=\{($x$1,$ y$1),($x$2,$ y$2),...,($xn$,$ yn$)\}$ where the

feature vectors are xi in Rd and yi in $\{0,1\},$ i $=1,2,...,$ n$.$ Note that in the lectures we assumed

yi in $\{1,+1\}.$

$1.$ Show that

P$[$y$|$x; w$]=$ P$[$y $=1|$x; w$]$y $$ P$[$y $=0|$x; w$](1$y$)(1)$

$2.$ Following the derivation of logistic regression in lectures, derive the log$-$likelihood for the training

data when the label of each training example is set to be yi in $\{0,1\}$ and sigmoid function is used

to covert the linear predictions to probabilities.

$3.$ Then, calculate the gradient, and write down the gradient descent $($GD$)$ update rule for the obtained

optimization problem and discuss the contribution of each training example to updated solution in

every iteration of GD$.$ In particular, compare the contribution of a misclassified example with the

contribution of a correctly classified example to the gradient.