3. (13 points - Correctness) Entropy and expected coding length (a) (2 points - Correctness) Consider the following two distributions defined by the counts below, where the numbers represent the counts of 5 different values of a random variable. Distribution 1:(a,4.5.3,2) Distribution 2: 2,2,39,8,1) Which of the above two distributions has a higher entropy and why? You should NOT give the numerical entropy here (though you are welcome to compute it to check that your reasoning matches the math). Your explanation should be a qualitative argument. (b) (2 points - Correctness) Calculate the entropy for the given distribution. Show your work for full credit. Symbol Counts 12 6 3 3 6 (c) (4 points - Correctness) Consider a text file with the following content: nnnnaameemmmnnamamnn Find the expected length (number of bits) of a coded symbol under each of the following coding schemes, using the distributions of the symbols in the above file. Show your work for full credit. Coding Scheme 1 Coding Scheme 2 Symbol CodeWord Symbol CodeWord 00 n 01 a 10 m 11 e 0 110 101 (d) (5 points - Correctness) Consider the following distribution Symbol p(symbol) 0.26 0.31 0.16 0.22 0.05 (i) Use Huffman's algorithm to derive an optimal code for this distribution, and then give the expected length (number of bits) of a coded symbol using this code and the distribution above. Show your work. (ii) The entropy for the distribution above is 2.148. Compare the value you calculated for the expected length of a coded symbol to the value of the entropy of the distribution. Is it is same, higher or lower? Explain why