Problem #$3$:

$(1$ point$)$ Logistic regression is a widely used algorithm for binary classification problems.

Consider a binary classification problem where the labels yi in $\{0,1\}$ for i $=1,...,$ n and

the feature vectors are $\{$xi$\}$

n

i$=1$

$.$ The logistic regression model estimates the probability

that yi $=1$ given xi as:

P$($yi $=1|$ xi$)=\backslash$sigma $($w

$$xi $+$ b$)=1$

$1+$ e$($w$$xi$+$b$)$

$,$

where $\backslash$sigma $($z$)$ is the sigmoid function.

$$ Derive the logistic loss function $($negative log$-$likelihood$)$ for a single training

$1$

example $($xi

$,$ yi$).$

$$ Extend this to derive the empirical risk for the entire training dataset.Problem #$3$:

$(1$ point$)$ Logistic regression is a widely used algorithm for binary classification problems.

Consider a binary classification problem where the labels $y_{i}in\{0,1\}$ for $i=1,$dots,$n$ and

the feature vectors are $\{x_i{\}}_i=1^n.$ The logistic regression model estimates the probability

that $y_i=1$ given $x_i$ as:

$P(y_i=1|x_i)=(w^{TT}{x}_i+b)=\frac{1}{1+e^{-(w^{TT}{x}_i+b)}},$

where $(z)$ is the sigmoid function.

Derive the logistic loss function $($negative log$-$likelihood$)$ for a single training

example $(x_i,y_i).$

Extend this to derive the empirical risk for the entire training dataset.