Exercise $1.$ Separating two GaussiansExercise $2.$

Define a zero$-$mean Gaussian function as

$(x$;$0,{}^2)=\frac{1}{\sqrt[\phantom{2}]{2{}^2}}$exp$\{-\frac{x^2}{2{}^2}\}.$

We want to approximate the Gaussian function using a boxcar function

$(x$;$W)=\{\begin{array}{cc}\frac{1}{W},& -\frac{W}{2}x\frac{W}{2},\\ 0,& \text{o}\text{t}\text{h}\text{e}\text{r}\text{w}\text{i}\text{s}\text{e}\text{.}& \end{array}$

Find the best $W$ that can achieve this. Hint: Go to Wikipedia and learn the concept of KL$-$divergence.

Suppose you have two Gaussians located at $$ and $-$ respectively, i$.$e$.,$ Gaussian$(x$;$,{}^2)$ and Gaussian$(x$;$-,{}^2).$

We know that if $$ is large enough, then the two Gaussians will be separated. We also know that if $$ is

small enough, the two Gaussians will be separated. Fix $.$ What is the largest $$ you can have before the

two Gaussian become merged together? By merge, I mean that you will no longer be able to see the two

peaks. The figure below is an example for $=0\mathrm{.}5.$ As you can see, when $=0\mathrm{.}5,$ the two Gaussians cannot

be distinguished. Use paper and pencil to answer this question, and use Python$/$MATLAB to verify your

answer.