$[$Logistic Regression$](8$ pts$)$

Figure $1$: Data for Logistic Regression Question

Let the data distribution, as shown in Figure$1,$ represent the binary classification

problem where we fit the model p$($y$=1|$x$,\backslash$theta $)=\backslash$sigma $(\backslash$theta $\_(0)+\backslash$theta $\_(1)$x$\_(1)+\backslash$theta $\_(2)$x$\_(2)).$ As seen in class,

we do this by minimizing the negative log loss $($same as maximizing the likelihood$),$ as

shown below:

L$(\backslash$theta $)=-$l$(\backslash$theta $,$D$\_($train $))$

where l$(\backslash$theta $,$D$\_($train $))$ represents the log likelihood on the training set.

For the questions below, submit the answer to each question as a separate figure. We

just expect an approximation of the figures if you submit hand$-$drawn solutions, also

be careful about the clarity of your submitted figures.

$($a$)$ Show a decision boundary that possibly would correspond to hat$($w$)($final weights$)$

after training the regressor. How many datapoints are wrongly classified on the

training data?

$($b$)$ For this part, consider that a strong regularization is applied to the $\backslash$theta $\_(0)$ parameter

and we minimize

L$\_(0)(\backslash$theta $)=-$l$(\backslash$theta $,$D$\_($train $))+\backslash$lambda $\backslash$theta $\_(0)^(2)$

Since we apply a strong regularization, assume that $\backslash$lambda is a very large, so$,\backslash$theta $\_(0)$ is

pulled down all the way to $0,$ but all other parameters are unregularized. Show

a decision boundary that possibly would correspond to hat$($w$)\backslash$theta $\_(0)+\backslash$theta $\_(1)$x$\_(1)+\backslash$theta $\_(2)$x$\_(2)$ when $\{$:x$\_(1)=$x$\_(2)=0]$

$($c$)$ Now, heavy regularization is performed only on the $\backslash$theta $\_(1)$ parameter, i$.$e$.,$ we minimize

L$\_(1)(\backslash$theta $)=-$l$(\backslash$theta $,$D$\_($train $))+\backslash$lambda $\backslash$theta $\_(1)^(2)$

Show a decision boundary that possibly would correspond to hat$($w$).$ How many

datapoints are wrongly classified on the training data?

$($d$)$ Finally, heavy regularization is done only on the w$\_(2)$ parameter. Show a decision

boundary that possibly would correspond to hat$($w$).$ How many datapoints are wrongly

classified on the training data?