$4)$ Information entropy. $($Computer science, Mathematics, Complexity$)(2)$ Entropy is a measure of your ignorance about a system; it is a measure of the lack of information. It has important implications in communication technologies: messages passed across a network communicate information, reducing the information entropy for the receiver. Your grandparent has sent you an e$-$mail message. From the header of the message, you know it contains $1000$ characters. You know each character is made of $8$ bits, which allows $2^8=256$ different letters or symbols per character. $($a$)$ Assuming all possible messages from your grandparent are equally likely $($a typical message would then look like G$^*$ me$\text{'}!8$V$[$beep$]...),$ how many different messages N could there be$?$ What is the corresponding upper bound S$\_$max for the information entropy k$\_$Slog N $?$ Your grandparent writes rather dull messages; they all fall into the same pattern. They have a total of $16$ equally likely messages. $^54$ After you read the message, you forget the details of the wording anyhow, and only remember these key points of information. $($b$)$ What is the actual information entropy change $\backslash$Delta S$\_$Shannon you undergo when reading the message? If your grandparent writes one message per month, what is the minimum number of $8-$bit characters per year that it would take to send your grandparent's messages? $($You may lump multiple messages into a single character.$)($Hints: $\backslash$Delta S$\_$Shannon is your change in entropy from before you read the message to after you read which of $16$ messages it was. The length of $1000$ is not important for this part.$)$ Remark: This is an extreme form of lossy data compression, like that used in jpeg images, mpeg animations, and mp$3$ audio files. We are asking for the number of characters per year for an optimally compressed signal.