[13] Suppose we have a random variable X that can assume values

a, b,

c, d with probabilities 1 2 , 1 4 , 1 8 , and 1 8 , respectively, with no dependency between the consecutive occurrences.

(a) Show that the entropy H(X) is 7 4 (bits per symbol).

(b) Show that the code E with E

(a) = 0, E

(b) = 10, E

(c) = 110, and E

(d) = 111 achieves this limiting value.