Section $1.$ Entropy

You are playing an on$-$line card game that uses a deck of $100$ cards containing:

$3$ Aces

$7$ Kings

$25$ Queens

$31$ Jacks

$34$ Tens

In each round of the game the cards are shuffled, you make a bet about what type of card will be drawn,

then a single card is drawn and the winners are paid off. The card is reinserted into the deck before the next

round begins.

$1\mathrm{.}1$ How much information do you receive when told that a Queen has been drawn during the current round?

$1\mathrm{.}2$ Give a numeric expression for the average information content received when learning about the

outcome of a round. The value of the numeric expression is $1\mathrm{.}97.$

$1\mathrm{.}3$ Construct a variable$-$length Huffman encoding that minimizes the length of messages that report the

outcome of a sequence of rounds. The outcome of a single round is encoded as A $($ace$),$ K $($king$),$ Q $($queen$),$

J $($jack$)$ or X $($ten$).$ Specify your encoding for each A$,$ K$,$ Q$,$ J and X$.$ Using your code, what is the expected

length of a message reporting the outcome of $1000$ rounds $($i$.$e$.,$ a message that contains $1000$ symbols$)?$

$1\mathrm{.}4$ Same question as the previous one but using Shannon$-$Fano instead of Huffman.

$1\mathrm{.}5$ The Monaco Gaming Commission regularly received messages in which the outcome for each round is

encoded using the symbols A$,$ K$,$ Q$,$ J and X$.$ They discover that a large number of messages describing the

outcome of $1000$ rounds $($i$.$e$.$ messages with $1000$ symbols$)$ can be compressed by the LZW algorithm into

files of size $43$ bytes in total. They immediately issue an indictment for running a crooked game. Briefly

explain their reasoning.