1. logistic regression

a. Logistic function and logit

Show: $$p = \frac{1}{1 + \exp\left\{-\mathbf{x}'\beta ight\}} \quad\Longleftrightarrow\quad \log\left(\frac{p}{1 - p} ight) = \mathbf{x}'\beta$$

b. Logistic regression likelihood

Let $Y_i \sim \text{Bernoulli}(p_i)$ for $i = 1, \dots, n$, assume $Y_i\perp Y_j$ for $i eq j$. i) Note that the probability mass function for $Y_i$ is $f(y_i; p_i) = p_i^{y_i}(1 - p_i)^{1 - y_i}$. Write the log-likelihood function: $$\ell(\mathbf{p};\mathbf{y}) = \log\mathcal{L}(\mathbf{p};\mathbf{y}) = \log\left(\prod_{i = 1}^n f(y_i; p_i) ight)$$

ii) Denote the contribution of the $i$th term to the log-likelihood by $\ell_i = \log\left(f(y_i) ight)$. Reparametrize $\ell_i$ in terms of $\theta_i = \log\left(\frac{p_i}{1 - p_i} ight)$, and write the log-likelihood of the data as a function of $\theta$, *i.e.*, write: $$\ell(\theta;\mathbf{y}) = \log\left(\prod_{i = 1}^n f(y_i; \theta_i) ight)$$

iii) Let $\mathbf{X}$ be an $n\times p$ matrix with rows $\mathbf{x}_1, \dots, \mathbf{x}_n$ and suppose that for each $i$: $$\log\left(\frac{p_i}{1 - p_i} ight) = \mathbf{x}_i'\beta$$ Write the log-likelihood of the data in terms of $\beta$.

c. Latent interpretation

Let $Z = \mathbf{x}'\beta + \epsilon$ where $\epsilon \sim \text{logistic}(0, 1)$ and $\mathbf{x}$ is fixed. Define $Y = I(Z > 0)$. Show: $$P(Y = 1) = \frac{1}{1 + \exp\{-\mathbf{x}'\beta\}}$$