It can also be shown that e plws i, o) = [ N (w;0,02E507) pl;|a)dE; (1.32) Jo where we introduced another variable ; (0, +00). Let v = 1. Eq. 1.32 can be further simplitied to plwlr,a=1)= [ N (w;]0,07E507) 3 exp (%) df; {1.33) Ja Hence, 1 1 w? ( [, 00 = 1) x expd | _ +; 1.34 p(; |y ) T, I { 3 (j!fz'f E})} (1.34) For simplicity, in the rest of this section, we fix o = 1, and remove the dependence on . People may wonder what is the point of getting conditional distributions of A; and ;. As you might be aware of, the optimisation of the posterior p(w|t, i, o) is not a easy task, while that of the complete-data posterior p{w|A .t, 11, ox) is much easier. This calls for the use of EM algorithns, where A, are latent variables. In lectures, we learnt how to apply EM to likelihoods. Here, we introduce how to apply EM to posteriors. Definition 2 (EM algorithm on posteriors). Consider the posterior defined by p(8|X), where @ is the parameter and X is the dataset. We want to perform MAP on p(@|X], but p{@|X) itself can be intractable. Instead, we consider the complete log-data posterior log p(@| X, Z) where Z is the set of latent variables. Similar to EM on likelihoods, we want to maximise the expectation of the log posterior with respect to the latent posterior p{Z|X, #). The detailed steps for performing an EM update is shown below. Initialisation: set ' E-step: evaluate p(Z|X, 871%) M-step: evaluate #% given by 8" = arg max (6, 0'), where @ Q(8,8) = /p(zp:, 0) log p(6|X., Z)dZ (1.35) JE Update: 89 + guew Question 1.7: EM update on Bayesian SVIM (15 Points) Perform an EM update of Bayesian SVM for the special case where a = 1. We define a set of variables: (1) Design matrix, BV =M (2) Target, t BV (3) Weight covariance, = diag(crj',?}J':L__M (4) Latent variables, X RN RY (5) Latent variable matrices, A = diag(X;);1 n, E = diag(gj)_,-=|___,u (6) and = 0. To do this, calenlate the complete data posterior p(@|X, Z), then take the logarithm. The complete data posterior should be a Gaussian A/(w|b, B~!). Represent b, B~! using only @, t, , A, A, E. The maximisation of the expectation of the complete log-posterior can then be found by substituting the conditional expectation of A and , denoted as X and , back to b. We will not be deriving those equations here due to page limitation. The task for you here is just deriving the posterior. Following the EM update on the posterior until convergence, we will approach the solution to an SVM