Please can you answer part (b)

Now, let us try to learn the parameterisation of the joint distribution using the infor- (b) [6 marks] mation gleaned from the missing data. Demonstrate that the result of the t-th E-step of the EM algorithm, based on the lower bound given above, is given by the following expressions: With this in mind we may write the log-likelihood function as: qin (12 = j) =- for: i, j = 0 or 1 L = log 200 10 201 0711 ain non nih Thi (*1 = j) = at = 1 + atal for: i, j = 0 or 1 X _ px(x1 = 0, 12) X E px(21 = 1, 12) x2 6 {0,1} x26{0,1} nho (c) [7 marks] X E px(21, 12 = 0) X E px(21, 22 = 1) Demonstrate that the result of the t-th M-step of the EM algorithm is given by the 216{0,1} x16{0,1} following expressions: Where: n = n - nhh. [NB: Again, you should handle parameter constraints using the following re-parameterisation In forming this log-likelihood function we have 'summed over' the missing, or hidden, trick: akl = enkl Eke"ki , Where nki E R.] data, which we treat as the outcome of hidden variables. alj = = (nij + nindin (22 = j) + nnidhi(21 = 1)) for: i,j = 0 or 1 Maximising this function looks like a job for the EM algorithm. Using similar reasoning to that given in the lecture slides when we encountered this algorithm (albeit in a different context), we can obtain the following lower bound on the function: (d) [6 marks] L > noo log woo + n10 log a10 + no1 log a01 + nii log all You are presented with survey data from a particular year: + non E qoh(12) log Px (21 = 0, 12) 226{0,1} qoh (2) Answers to 2nd Question: Like Maths Dislike Maths Not Answered + nin E qin(12) log Px (21 = 1,22) Answers to Like Machine 150 25 78 x26 {0,1} qin (202) Learning _ gho(21 ) log Px ($1,22 = 0) 1st Question: Dislike Machine 32 13 62 + nho 1E{0,1} gho (21) Learning Not Answered 45 25 82 + nhi E ahi (21) log Px (21, 22 = 1) ciE {0,1} ghi (21 ) Given this data and assuming an initialisation such that al1, @10, 291, @80 are set equal to the estimates derived in part (a) of this question, then calculate parameter es- Here qoh(.), qih(.) are pdf's over x2, and qho(.), ghi( ) are pdf's over 21. timates at convergence (to 2 decimal places) of the EM algorithm, i.e. calculate all, "10, 01, 200 to two decimal places as t becomes sufficently large. In addition, The EM algorithm seeks to learn the parameters: 200, @10, 201, a11, doh (22), 91h(202), for this level of accuracy, state what value of t constitutes 'large' in this case. Tho (21), ghi (1) (for x2 E {0, 1} and x1 E {0, 1}) iteratively. After the t-th iteration of the algorithm the estimates of these parameters are denoted by: 200, @10, 201, @11, don (2-2), 9in (22), Tho(21), Thi (21)