Consider a discrete memory less source with source alphabet L {s₀, s₁,. . . , s_k–1) and source statistics (p₀, p₁, ... p_k–1,) The nth extension of this source is another discrete memory less source with source alphabet Lⁿ’ = {σ₀, σ₁, .., σ_{M – 1}), where M  = Kⁿ. Let P(σ_i) denote the probability of σ_i.

(a) Show that which is to be expected.

(b) Show that where p_ik is the probability of symbol s_ik, and H(L) is the entropy of the original source.

(c) Hence, show that