Problem (1): (10 points) (a) The alphabet size for a r.v. Xis 27 = n. Let p(x) be the PMF for X and u(x) be the PMF of a uniform distribution over the same alphabet E. Prove that the Kullback-Leibler distance between the two PMF's is: D(p||u) = log n - H(X) (b) Given two r.v's X and Y over the same alphabet: Show that the bounds of Mutual Information (MI) are 0 SI(X:Y) Smin [ H(X), H(Y)] with equality on the left if and only if X and Y are independent random variables, and with equality on the right if and only if either Y essentially determines X, or X essentially determines Y , or both. Problem (1): (10 points) (a) The alphabet size for a r.v. Xis 27 = n. Let p(x) be the PMF for X and u(x) be the PMF of a uniform distribution over the same alphabet E. Prove that the Kullback-Leibler distance between the two PMF's is: D(p||u) = log n - H(X) (b) Given two r.v's X and Y over the same alphabet: Show that the bounds of Mutual Information (MI) are 0 SI(X:Y) Smin [ H(X), H(Y)] with equality on the left if and only if X and Y are independent random variables, and with equality on the right if and only if either Y essentially determines X, or X essentially determines Y , or both