Let X1, . . . , Xn | iid Multinomial(1,) on k elements with a similar notation as seen in class: the encoding for a possible value xi of the random vector Xi is xi = (x (i) 1 , x (i) 2 , . . . , x (i) k ) with x (i) j {0, 1} and Pk j 0=1 x (i) j 0 = 1 (that is, we have a j where x (i) j = 1 and for each j 0 6= j , x (i) j 0 = 0). Consider a Dirichlet prior distribution on : Dirichlet(), where = (1, 2, . . . , k) and j > 0 for all j. (The Dirichlet distribution is a distribution for a continuous random vector which lies on the probability simplex k. Recall k := { R k : 0 j 1 and Pk j=1 j = 1}. Its probability density function1 is p(|) = (Pk j=1 j ) Qk j=1 (j ) Qk j=1 j1 j . Note that the beta distribution seen is class is the special case of a Dirichlet distribution for k = 2, like the binomial distribution is the special case of a multinomial distribution for k = 2.)

(a) Supposing that the data is IID, what are the conditional independence statements that we can state for the joint distribution p(, x1, . . . , xn)? The answer should be in the form of formal conditional independence statements.

(b) Derive the posterior distribution p( | x1, . . . , xn).

(c) Derive the marginal probability p(x1, . . . , xn) (or equivalently p(x1, . . . , xn | ).) This quantity is called the marginal likelihood and we will see it again when doing model selection later in the course.

(d) Derive the MAP estimate for assuming that the hyperparameters for the Dirichlet prior satisfy j > 1 for all j. Compare this MAP estimator with the MLE estimator for the multinomial distribution seen in class: what can you say when k is extremely large?