Question: 9 Points [Entropy] (a) (3 points) Show that, for a source generating N symbols, the entropy is maximum when the symbols are all equiprobable. (b)

9 Points [Entropy]

(a) (3 points) Show that, for a source generating N symbols, the entropy is maximum when the symbols are all equiprobable.

(b) (5 points) Suppose we receive the sequence of symbols 4 5 8 6 4 7 8 9 4 8.

i. If we know that the probabilities of receiving such symbols are: P4 = P5 = P6 = P7 = P8 = P9 = 1 6, then what is the entropy of symbols under this probability model?

ii. Now assume that the probability model is given by the frequency of occurrence of the symbols that were actually received. Now what is the entropy? Explain the di?erence.

iii. Calculate how many bits in total and how many bits per symbol are needed to transmit symbols in the probability model stated above in part (ii) if the actual symbol stream is: 4 4 4. Explain any discrepancy from the entropy.

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer

Step: 1 Unlock blur-text-image

Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock

Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!

ttth Suppose that the sequence of bags {Bn | n N} is recursively enumerated by the computable function e(n, x) = fn(x), [7 marks] Hence prove that the set of all recursive bags cannot be recursively...

Portray in words what transforms you would have to make to your execution to some degree (a) to accomplish this and remark on the benefits and detriments of this thought.You are approached to compose...

A creative engineer suggests structuring the TLB so that not all the bits of the presented address need match to result in a hit. Suggest how this might be achieved, and what might be the costs and...

using Visual Community Studio 2022 and there are few errors. Please insert of examples as well for me to understand more clearly as well. Thank you. The code: #include using namespace std; int...

CANMNMM January of this year. (a) Each item will be held in a record. Describe all the data structures that must refer to these records to implement the required functionality. Describe all the...

Focus: It is depending on the shape and structure of A: tall, square, symmetric. For each technique, give the relevant matrix equations to obtain the solution x, and point out the properties of the...

Have a C compiler which is ANSI conforming in all respects except that it has no facility for the definition, declaration or use of standard C structures. Outline a set of routines written in this...

An example program using a future is shown below. Future f = new Future() { @Override public String execute() { // ...long running computation... return data; }; // ... String result = f.get(); //...

re Regular Languages and Finite Automata (a) Let L be the set of all strings over the alphabet {a, b} that end in a and do not contain the substring bb. Describe a deterministic finite automaton...

Represent multisets using binary trees. Compare the representations (a), (b) and (c), stating the advantages and drawbacks of each. [Answers need not contain ML code. You may refer to algorithms and...

Trust is a significant element for voluntary distributed computing, especially because users are warned about the dangers of downloading executable programs to their computers from unknown sources....

Even without picking up your pencil you can probably determine at least one conclusion about the data in Exercise 17.13. What is that conclusion, and why is it of no interest?

Suppose the U . K . interest rate is 1 4 % and the U . S . dollar rate is 1 0 . 5 % . According to the IRP, what should be the forward premium on the dollar?

Seved Help 14 Wisconsin Snowmobile Corp. is considering a switch to level production Cost efficiencies would occur under level production, and aftertax costs would decline by $31,500, but inventory...

1. What can HR professionals do to mitigate concerns about pregnancy discrimination in the workplace? What does the law say companies need to do to avoid violating the rights of pregnant employees?

2. If you were an HR manager at UPS, how would you have handled the situation described in the case? How could the employees immediate manager have handled the situation differently?

LO2 Identify components of workflow analysis.