. Let the random variable X be five possible symbols {a, ,y, 8, e). Consider two...

Transcribed Image Text:

. Let the random variable X be five possible symbols {a, ß,y, 8, e). Consider two probability distributions p(x) and q(x) over these symbols, and two possible coding schemes C₁ (x) and C₂ (x) for this random variable: Symbol p(x) q(x) C₁(x) 1/2 1/2 0 10 α B Ύ 8 € C₂(x) 0 100 1/4 1/8 1/8 1/8 110 101 1/16 1/8 1110 110 1/16 1/8 1111 111 (a) Calculate H(p), H(q), D (p|lq) and D(q|lp). (b) The last two columns above represent codes for the random variable. Is code C₁ optimal for p(x)? Is code C₂ optimal for q(x)? (c) Assume that we use code C₂ when the distribution is p. What is the average length of the codewords? By how much does it exceed the entropy H(p)? Relate your answer to D (pl|q). (d) Assume that we use code C₁ when the distribution is q. What is the average length of the codewords? By how much does it exceed the entropy H(q)? Relate your answer to D(qllp). . Let the random variable X be five possible symbols {a, ß,y, 8, e). Consider two probability distributions p(x) and q(x) over these symbols, and two possible coding schemes C₁ (x) and C₂ (x) for this random variable: Symbol p(x) q(x) C₁(x) 1/2 1/2 0 10 α B Ύ 8 € C₂(x) 0 100 1/4 1/8 1/8 1/8 110 101 1/16 1/8 1110 110 1/16 1/8 1111 111 (a) Calculate H(p), H(q), D (p|lq) and D(q|lp). (b) The last two columns above represent codes for the random variable. Is code C₁ optimal for p(x)? Is code C₂ optimal for q(x)? (c) Assume that we use code C₂ when the distribution is p. What is the average length of the codewords? By how much does it exceed the entropy H(p)? Relate your answer to D (pl|q). (d) Assume that we use code C₁ when the distribution is q. What is the average length of the codewords? By how much does it exceed the entropy H(q)? Relate your answer to D(qllp).