You will be tasked with implementing a K$-$means clustering and a Gaussian mixture model using Gibbs sampling. For the case of

the Gaussian mixture model, suppose our data generating process is:

$x_{i}N({}_{z_i},{}_0^{2}I)$

${}_{k}N({}_0,{}_0^2)$

$z_{i}Categorical()$

$Dirichlet({}_0)$

You may fix the value of the hyperpriors ${}_0^2,{}_0^2,{}_0,{}_0.$ Note that the likelihood for $x_i$ is a multivariate Gaussian distribution.

$1.)$ Derive the posterior updates for the parameters ${}_k,,z_i.$ For deriving the full conditional of ${}_k,$ two properties that may be

useful are the properties of the conditional distribution of a m$.$v$.$ Gaussian and completing the square of a matrix.

$2.)$ Compare the Gibbs sampling algorithm using the posterior updates you wrote in Question $1$ with the K$-$Means algorithm.

Under what conditions is the K$-$Means algorithm a special case of the Gibbs sampler for a GMM$?$

$3.)$ Now, implement the Gibbs sampler for the GMM and the K$-$Means algorithm. Test to see if your implementation works

correctly by fitting the model by generating some synthetic data. You may use the function 'sklearn.datasets.make$\_$blobs' with

the default setting to test this. One quantitative way to measure the performance per iteration to track the progress of your

model is to calculate the log$-$likelihood of the data per iteration. If it improves on average, then you may be in the right direction

$($this is not mandatory, but it should help during the debugging process$).$

$4.)$ Fit the data to the 'sklearn.datasets.load$\_$digits' handwritten data set. It may be helpful to rescale the data to have zero mean

and unit variance. Try to plot the cluster centers, comment on the performance.