(Kullback Leibler divergence between discrete random variables:) Let X and Y be two discrete random variables...

Transcribed Image Text:

(Kullback Leibler divergence between discrete random variables:) Let X and Y be two discrete random variables with non-zero pmf defined on the set of integers {1,2,...,m}. (a) Show that the function g(x) = ln x − x + 1 ≤0. (Hint: Show that g"(x) < 0 and find its unique maxima to establish the inequality.) (b) Using In x < x-1 from part (a), show that, m Σpx(i) In Px (i) i=1 ≥ 0, py (i) (13) with equality occuring only if px (i) = py(i) for all i = 1,2,..., m. (Hint: Work with the negative of expression on the LHS of the inequality.) (Kullback Leibler divergence between discrete random variables:) Let X and Y be two discrete random variables with non-zero pmf defined on the set of integers {1,2,...,m}. (a) Show that the function g(x) = ln x − x + 1 ≤0. (Hint: Show that g"(x) < 0 and find its unique maxima to establish the inequality.) (b) Using In x < x-1 from part (a), show that, m Σpx(i) In Px (i) i=1 ≥ 0, py (i) (13) with equality occuring only if px (i) = py(i) for all i = 1,2,..., m. (Hint: Work with the negative of expression on the LHS of the inequality.)

Answer rating: 100% (QA)

The detailed ... View the full answer