We would like to compute entropy, joint entropy and conditional entropy

using Python $($Matlab$,$ R$,$ or any other programming language of your

choice$).$ Please include code snippets:

$($a$)$ Write code for computing the entropy of a binary variable and plot

it as a function of the probability $p.$

$($b$)$ Compute the joint entropy for a simple two random variables example

with:

$p(x,y)=\{\begin{array}{ccc}0,& 0& w.p.\frac{1}{4}\\ 0,& 1& w.p.\frac{1}{4}\\ 1,& 0& w.p.\frac{1}{3}\\ 1,& 1& w.p.\frac{1}{6}\end{array}$

$($c$)$ Alternatively, compute the joint entropy of $(1)$ through the calcula$-$

tion of $H(x)$ and $(x|),$ and applying the entropy chain rule.

We would like to compute entropy, joint entropy and conditional entropy

using Python $($Matlab$,$ R$,$ or any other programming language of your

choice$).$ Please include code snippets:

$($a$)$ Write code for computing the entropy of a binary variable and plot

it as a function of the probability $p.$

$($b$)$ Compute the joint entropy for a simple two random variables example

with:

$p(x,y)=\{\begin{array}{ccc}0,& 0& w.p.\frac{1}{4}\\ 0,& 1& w.p.\frac{1}{4}\\ 1,& 0& w.p.\frac{1}{3}\\ 1,& 1& w.p.\frac{1}{6}\end{array}$

$($c$)$ Alternatively, compute the joint entropy of $(1)$ through the calcula$-$

tion of $H(x)$ and $(x|),$ and applying the entropy chain rule.