we talked about Huffman coding and Shannon's source coding theorem for DMS(Q). Both Huffman coding and...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
we talked about Huffman coding and Shannon's source coding theorem for DMS(Q). Both Huffman coding and Shannon's source coding compress an i.i.d. source by using the knowledge of the distribution of the source. In this problem, we consider the case when the true distribution Q of the source is unknown. For such a case, we will design a universal code' with which we can compress the source without the knowledge of the exact distribution Q. To design such a universal code, we first introduce some definitions. Consider an i.i.d. source X₁, X2,..., Xn distributed by distribution Q. An (n, R) block source code of rate R consists of a pair of mappings fn and gn: fn X→ {0, 1}R In: (0,1}"X". The probability of error for the code with respect to the distribution Qis p(n) Q ({z": gn (fn(x")) #x"}). (1) (2) We say that an error exponent E is achievable at rate R if there exists a sequence of (n, R) codes with P(n) e-nE (3) which means 1 lim sup - In P() <-E. (4) 14x n Let's design a universal code (fn, 9n) that enumerates all the sequences in the set A = {r" € X": H(P₂) ≤S} (5) for some constant S > 0, where P is the empirical distribution of each r". The encoder fn maps each sequence z E A to different codewords by one-to-one mapping, and the decoder gn recovers the sequence by the inverse mapping of fn. Note that this code does not assume or use the distribution of the source. Error is claimed whenever r" # A. The error probability of this code for DMS(Q) is thus P(n) Q ({x":" & A}). (6) a) Find the required rate R for the encoder fn to enumerate all the sequences in the set A. (Hint: Find the size |A|.) b) We now show that this encoding scheme is universal. Assume that the distribution of X₁, X2,..., X₂ is Q and H(Q)< S. Find the error exponent of this code, i.e., find E in (4) for P) in (6). (In your answer, if you have any optimization, then just leave it. You don't need to solve the optimization.) we talked about Huffman coding and Shannon's source coding theorem for DMS(Q). Both Huffman coding and Shannon's source coding compress an i.i.d. source by using the knowledge of the distribution of the source. In this problem, we consider the case when the true distribution Q of the source is unknown. For such a case, we will design a universal code' with which we can compress the source without the knowledge of the exact distribution Q. To design such a universal code, we first introduce some definitions. Consider an i.i.d. source X₁, X2,..., Xn distributed by distribution Q. An (n, R) block source code of rate R consists of a pair of mappings fn and gn: fn X→ {0, 1}R In: (0,1}"X". The probability of error for the code with respect to the distribution Qis p(n) Q ({z": gn (fn(x")) #x"}). (1) (2) We say that an error exponent E is achievable at rate R if there exists a sequence of (n, R) codes with P(n) e-nE (3) which means 1 lim sup - In P() <-E. (4) 14x n Let's design a universal code (fn, 9n) that enumerates all the sequences in the set A = {r" € X": H(P₂) ≤S} (5) for some constant S > 0, where P is the empirical distribution of each r". The encoder fn maps each sequence z E A to different codewords by one-to-one mapping, and the decoder gn recovers the sequence by the inverse mapping of fn. Note that this code does not assume or use the distribution of the source. Error is claimed whenever r" # A. The error probability of this code for DMS(Q) is thus P(n) Q ({x":" & A}). (6) a) Find the required rate R for the encoder fn to enumerate all the sequences in the set A. (Hint: Find the size |A|.) b) We now show that this encoding scheme is universal. Assume that the distribution of X₁, X2,..., X₂ is Q and H(Q)< S. Find the error exponent of this code, i.e., find E in (4) for P) in (6). (In your answer, if you have any optimization, then just leave it. You don't need to solve the optimization.)
Expert Answer:
Related Book For
Building Java Programs A Back To Basics Approach
ISBN: 9780135471944
5th Edition
Authors: Stuart Reges, Marty Stepp
Posted Date:
Students also viewed these databases questions
-
Write the balanced molecular ionic and net ionic equations for each of the following - A) Nitric Acid + Iron (III) Hydroxide > B) Chromic Acid + Sodium Hydroxide > Water + Sodium Chromate We...
-
Determine V. for the given clipper creui't anl sketeh it. Via 20vt Vo 8V Ideal -20V
-
Suppose the (electrically neutral) y z plane carries a time-dependent but uniform surface current K(t) z. (a) Find the electric and magnetic fields at a height x above the plane if (i) A constant...
-
Using the information from 2-16, create a cash flow statement for 2013. In 2-16 Bristle Brush-Off Corporation: Income Statements for Years Ended December 31 ($000s) 2013 2012 $7,950 5,100 350 750...
-
For Korb Company, actual sales are $1.2 million and break-even sales are $840,000. Calculate (a) The margin of safety in dollars and (b) The margin of safety ratio?
-
Determine whether the following statements are true and give an explanation or counterexample. a. If the curve y = f(x) on the interval [a, b] is revolved about the y-axis, the area of the surface...
-
Law of corresponding state was proposed and formulated by (a) R.S. Pitzer (b) Gibbs-Duhem (c) van der Waals (d) Charles Dickens.
-
Madson Corporations balance sheet at December 31, 2013, is presented below. During January 2014, the following transactions occurred. Madson uses the perpetual inventory method. Jan. 1 Madson...
-
(a) For each of the following integrals, determine if they are proper or improper. If an integral is improper, briefly explain why (one sentence), and write it as a limit (or sum of limits) of proper...
-
The lifetime (in months) of a battery is modeled by a random variable X that has pdf f ( x ) = K x 1 ( x > 0 ) where K = ln ( 1 / ) for an unknown parameter ( 0 , 1 ) . (Here 1 ( x > 0 )...
-
This problem is for the month of February for OverPriced Jeans (OPJ). You will be using the new Revenue and Expense accounts that you learned about in this chapter to record OPJ's February...
-
What are the components that are involved in establishing brand identity?
-
What is the difference between primary and secondary data? When should you use one versus the other?
-
What is the purpose of a creative brief?
-
Why should objectives, strategies and tactics be cascading?
-
Find three mission statements online and analyze if they provide direction and motivation and address stakeholders.
-
1) A newly retired individual purchased a large plot of land in the Amazon jungle at an extremely low cost, with plan of making a small house and staying there. The major problem is that the nearest...
-
Akramin just graduated with a Master of Engineering in Manufacturing Engineering and landed a new job in Melaka with a starting salary of RM 4,000 per month. There are a number of things that he...
-
Add the following accessor methods to your Rectangle class: public boolean contains(int x, int y) public boolean contains(Point p) Returns whether the given Point or coordinates lie inside the bounds...
-
Write a method zoomIn that accepts a DrawingPanel as a parameter and converts it into an image twice as large in both dimensions. Each pixel from the original image becomes a cluster of 4 pixels (2...
-
Assume that you have a variable called count that will take on the values 1 , 2 , 3 , 4 , and so on. You are going to formulate expressions in terms of count that will yield different sequences. For...
-
A binary liquid mixture consists of \(60 \mathrm{~mol}\) per cent ethylene and \(40 \mathrm{~mol}\) per cent propylene. At \(423 \mathrm{~K}\), the vapour pressure of ethylene and propylene are...
-
Deduce the Gibbs-Duhem-Margules equation for a binary solution starting from Raoult's law.
-
The vapour pressure of acetone, acetonitrile and nitromethane can be represented by Antoine equations as \[ \begin{aligned} & \ln P_{1}^{\text {Sat }}=14.3916-\frac{2795.82}{T+230.0} \\ & \ln...
Study smarter with the SolutionInn App