2.14 ( ) www This exercise demonstrates that the multivariate distribution with maximum entropy, for a given covariance, is a Gaussian. The entropy of a distribution p(x) is given by H[x] = −



p(x) lnp(x) dx. (2.279)

We wish to maximize H[x] over all distributions p(x) subject to the constraints that p(x) be normalized and that it have a specific mean and covariance, so that



p(x) dx = 1 (2.280)



p(x)x dx = μ (2.281)



p(x)(x − μ)(x − μ)T dx = Σ. (2.282)

By performing a variational maximization of (2.279) and using Lagrange multipliers to enforce the constraints (2.280), (2.281), and (2.282), show that the maximum likelihood distribution is given by the Gaussian (2.43).