5. [5] Let E1 and E2 be two independent flips of a fair coin. Find the entropy H(E1) and joint entropy H(E1,E2). Why is H(E1,E2)=H(E1)+H(E2)?

1. Consider modifying the experiment in 5 above so that either 1) the coin is loaded OR 2) you only flip a second time when the first flip is heads, when tails you assign the value of the flip E2 to be tails. Which modification of the experiment (1 or 2) will make it so that H(E1,E2) < H(E1)+H(E2) ? Will one of them make it so H(E1,E2) > H(E1)+H(E2) ? 2. Let X be a random variable that takes on integer values. The probability is that X will be in the range [0,24-1] with all values in that range being equally likely. The rest of the time, it will be in the range [24,28-1], again with uniform probability. What is the entropy H(X)? (Estimation is OK).

3. A bag contains 7 red balls, 2 green balls, and 1 black ball. a. You choose 2 balls from the bag with replacement (i.e. you put the first ball back in and shake before drawing the second). What is the entropy of the experiment? b. You choose 2 balls from the bag without replacement (i.e. you keep the first ball while drawing the second). How is your uncertainty affected now that you know the color of the first ball before drawing the second? What is the entropy of this new experiment given the first ball you choose is red?

4. Write a small program that loads in a text file of any size and then prints the frequency (as a percentage) of each character (a..z). All characters should be made lowercase for counting purposes. Ignore punctuation, spaces, etc. Output should be sorted in decreasing frequency order (e, t, a, o, etc.) and look something like: e 13.27% t 9.11% a 8.47% o 7.32% q 0.01% z 0.00% Attach a printout of your code to your submission. Also, attach a printout of your code applied to THIS HANDOUT (ie: submit a screenshot of your results after analyzing the HW1 document). You may want to copy/paste the contents of this file into Notepad to create a text file your program can read.

use C++ language Program

Use C++ language program