~ ~ ~ (e) More recently, researchers have proposed to use the Wasserstein distance instead of...

Transcribed Image Text:

~ ~ ~ (e) More recently, researchers have proposed to use the Wasserstein distance instead of divergences to train the models since the KL divergence often fails to give meaningful information for training. Consider three distributions, P1 U[0, 1], P2 U[0.5, 1.5], and P3 U[1,2], where U[a, b] is uniform distribution over [a, b]. Calculate KL(P1, P2), KL(P1, P3), W1(P1, P2), and W(P1, P3), where W(,) is the Wasserstein-1 distance between two distributions. Hint: this subproblem requires no real mathematical computation. What you need to do is to understand the intuitive meaning of KL-divergence and Wasserstein distance. You may find wiki of Earth mover's distance and Wasserstein metric useful. ~ ~ ~ (e) More recently, researchers have proposed to use the Wasserstein distance instead of divergences to train the models since the KL divergence often fails to give meaningful information for training. Consider three distributions, P1 U[0, 1], P2 U[0.5, 1.5], and P3 U[1,2], where U[a, b] is uniform distribution over [a, b]. Calculate KL(P1, P2), KL(P1, P3), W1(P1, P2), and W(P1, P3), where W(,) is the Wasserstein-1 distance between two distributions. Hint: this subproblem requires no real mathematical computation. What you need to do is to understand the intuitive meaning of KL-divergence and Wasserstein distance. You may find wiki of Earth mover's distance and Wasserstein metric useful.