1. Maximum Likelihood Estimation Consider a random variable X (possibly a vector) whose distribution (density function or mass function) belongs to a parametric family. The density or mass function may be written f (a: ; 9), where 0 is called the parameter, and can be either a scalar or vector. For example, in the Gaussian family, 0 can be a two-dimensional vector consisting of the mean and variance. Suppose the parametric family is known, but the value of the parameter is unknown. It is often of interest to estimate this parameter from an observation of X. Maximum likelihood estimation is one of the most important parameter estimation techniques. Let X1, . . . , Xn be iid (independent and identically distributed) random variables distributed according to f (m; 9). By independence, the joint distribution of the observations is the product n Hf(mi; 9)- i=1 Viewed as a function of 0, this quantity is called the likelihood of 0. Because many common pdfs/pmfs have an exponential form, it is often more convenient to work with the log- likelihood, Dogma; 9). i=1 A maximum likelihood estimator (MLE) of 9 is a function 5 of the observed values 3:1, . . . ,mn satisfying n g E argmax Zlogxi; 9), 9 i=1 where \"arg max\" denotes the set of all values achieving the maximum. If there is always a unique maximizer, it is called the maximum likelihood estimator. (a) Let X1, . . . ,Xn iid bernoulli(6), a Bernoulli trial taking values 0 and 1, and \"success probability\" 6, where 1 corresponds to \"success\". Determine the maximum likelihood estimator of 6 as a function of observations (realizations) 3:1, . . . ,3", of X1, . . . ,Xn. (b) Calculate the Hessian of the log-likelihood function and verify that the estimator you found is indeed the unique maximizer of the log-likelihood function