Suppose that we have the data (tau_{mathrm{n}}=left{x_{1}, ldots, x_{n} ight}) in (mathbb{R}) and decide to train the

Suppose that we have the data \(\tau_{\mathrm{n}}=\left\{x_{1}, \ldots, x_{n}\right\}\) in \(\mathbb{R}\) and decide to train the two-component Gaussian mixture model

\[ g(x \mid \boldsymbol{\theta})=w_{1} \frac{1}{\sqrt{2 \pi \sigma_{1}^{2}}} \exp \left(-\frac{\left(x-\mu_{1}\right)^{2}}{2 \sigma_{1}^{2}}\right)+w_{2} \frac{1}{\sqrt{2 \pi \sigma_{2}^{2}}} \exp \left(-\frac{\left(x-\mu_{2}\right)}{2 \sigma_{2}^{2}}\right) \]

where the parameter vector \(\boldsymbol{\theta}=\left[\mu_{1}, \mu_{2}, \sigma_{1}, \sigma_{2}, w_{1}, w_{2}\right]^{\top}\) belongs to the set

\[ \Theta=\left\{\boldsymbol{\theta}: w_{1}+w_{2}=1, w_{1} \in[0,1], \mu_{i} \in \mathbb{R}, \sigma_{i}>0, \forall i\right\} \]

Suppose that the training is via the maximum likelihood in (2.28). Show that

\[ \sup _{\boldsymbol{\theta} \in \Theta} \frac{1}{n} \sum_{i=1}^{n} \ln g\left(x_{i} \mid \boldsymbol{\theta}\right)=\infty \]

In other words, find a sequence of values for \(\theta \in \Theta\) such that the likelihood grows without bound. How can we restrict the set \(\Theta\) to ensure that the likelihood remains bounded?