Let $Y$ be a random variable that takes values in $\{0,1\}.$ We assume that there exist

${}_0$inR and $in{R}^p$ such that the conditional probability $P(Y=1|x)$ given for

each $xin{R}^P$ can be expressed by

$log(\frac{P(Y=1|x)}{P(Y=0|x)})={}_0+x$

$($logistic regression$).$

$($a$)$ Prove that $(2\mathrm{.}22)$ can be expressed by

$P(Y=1|x)=\frac{\text{e}\text{x}\text{p}({}_0+x)}{1+\text{e}\text{x}\text{p}({}_0+x)}$

$($b$)$ Let $p=1$ and ${}_0=0.$ The following procedure draws the curves by com$-$

puting the right$-$hand side of $(2\mathrm{.}23)$ for various $inR.$ Fill in the blanks and

execute the procedure for $=0.$ How do the shapes evolve as $$ grows?

def f$($x$)$:

return $($np$.$axp$($beta$\_0+$ np$.$dot $($beta$,$ x$))$

$/(1+$ np$-$axp$($beta$\_0+$ np$.$dot $($beta$,$ x$))))$

beta$\_0=0$

beta$\_$seq $=$ np$.$array $([0,0\mathrm{.}2,0\mathrm{.}5,1,2,10])$

a $=$ len$($beta$\_$seq$)$

x $=$ np$.$aranga $(-10,10)$

for $1$ in range$($a$)$:

bata $=$ ## Blank$($l$)$ ##

plt$.$plot$($## Blank$(2)$ ##$,$ label$=$beta$\_$seg$[$ij$)$

plt$-$t$1$tle$("$Log$1$st$1$c$\_$Curve"$)$

plt$.$xlabal$("$$x$3")$

plt $-$ ylabol$("3$P$($Y$=1|$x$)3")$

P$1$t$.$ logend $()$

$($c$)$ For logistic regression $(2\mathrm{.}22),$ when the realizations $xin{R}^p$ and Yin$\{0,1\}$

are $(x_i,y_i)(i=1,$dots,$N),$ the likelihood is given by $pro{d}_i=1^N\frac{e^{y_i[{}_0+x_i]}}{1+e^{{}_0+x_i}}.$ In its

Lasso evaluation

$-\frac{1}{N}{\displaystyle \underset{i=1}{\overset{N}{}}}[y_i({}_0+x_i)-log(1+e^{{}_1+x_i})]+|||{|}_1$

$Y$ takes values in $\{0,1\}.$ However, an alternative expression as in $(2\mathrm{.}25)$ is

often used, in which $Y$ takes values in $\{-1,1\}$ : $\frac{1}{N}{\displaystyle \underset{i=1}{\overset{N}{}}}log(1+$exp$(-y_i({}_0+x_i)\})+|||{|}_1$

Show that if we replace $y_i=0$ with $y_i=-1,$ then $(2\mathrm{.}24)$ is equivalent to

$(2\mathrm{.}25).$

Hereafter, we denote by $xin{R}^{N(p+1)}$ the matrix such that the $(i,j)$ th element

is $x_{i,j}$ for $j=1,$dots,$p$ and the leftmost column $($the $0$ th column$)$ is a vector

consisting of $N$ ones, and let $x_i=[x_{i,1},$dots,$x_{i,p}].$ We assume that the random

variable $Y$ takes values in $\{-1,1\}.$