Variational Bayes: mixture model. Consider a mixture of normal model f(x | ) = PK k=1 kN(x | k, 2 ), with = (,

Variational Bayes: mixture model. Consider a mixture of normal model f(x |

θ) =

PK k=1

πkN(x | µk, σ2

), with θ = (π, µ, σ2

), where µ = (µ1, . . . , µK) and

π = (π1, . . . , πK). Let γ = 1/σ2

. We complete the model with conditionally conjugate priors h(µk) = N(0, τ), h(γ) = Ga

(a,

b) and π ∼ DK−1(α, . . . , α), with fixed hyperparameters (τ,

a, b, α) and prior independence.

a. Find the complete conditional posterior h(θ j

| θ−j

, x) for µk, πk and γ.

b. For the moment, fix σ

2 = 1 and π = (1/K, . . . , 1/K). Consider the meanfield variation family D with q(µk) = N(mk, sk), q(ci = k) = φik, and derive one iteration of CAVI, that is, find the updating equations for (mk, sk)

and φk j.
Hint: See the discussion in Blei et al (2017).

c. Now include γ and π in the parameter vector. Extend D by defining q(γ) = Ga

(c,

d) and q(π) = D(e1, . . . , eK). Find the updating equations for

c, d, and e.