Probability An Introduction 2nd Edition Geoffrey Grimmett, Dominic Welsh - Solutions

Explore comprehensive resources for "Probability An Introduction 2nd Edition" by Geoffrey Grimmett and Dominic Welsh. Access online answers key and solutions for solved problems, ensuring a thorough understanding of each chapter. Our solution manual offers step-by-step answers, perfect for both students and instructors. Dive into the test bank and textbook questions and answers to enhance your learning. With the instructor manual, chapter solutions, and solutions PDF, mastering probability concepts becomes straightforward. Download free resources and elevate your study experience with expertly crafted answers and solutions.

[37] The prefix-code E∗ of Example 1.11.16 is an asymptotically optimal universal code because E2 is already one.(a) Show that n 2−l∗(n) = c with c = 2.865064 ... . Show that the Kraft inequality, Theorem 1.11.1, is satisfied with equality for the length set ln = l∗(n) + logc, n = 1, 2,...
[26] (a) Show that the set of code words {x¯ : x ∈N} is a universal code-word alphabet. Show that it is not asymptotically optimal.(b) Show that the set of code words {l(x)x : x ∈ N} is a universal code-word alphabet. Show that it is also asymptotically optimal.Comments. Source: [P. Elias,
[22] Let a code have the set of code-word lengths l1, l2,... .Show that if the code is uniquely decodable, then the Kraft inequality, Theorem 1.11.1, must be satisfied.Comments. Thus, Theorem 1.11.1 holds for the wider class of uniquely decodable codes. This is called the McMillan–Kraft Theorem.
[19] Let K = {x : φx(x) < ∞} be the diagonal halting set(Section 1.7). Let k1, k2,... be the list of elements of K in increasing order. Define code E by E(x)=1kx 0.(a) Show that E is a prefix-code and uniquely decodable.(b) Show that E is not computable.(c) Show that the decoding function E−1
[09] (a) Define code E by E(x) = 01x for x = 1, 2,... . Show that E is uniquely decodable, but it is not a prefix-code.(b) Define a code E by E(x)=1x0 for x = 1, 2,... . Show that this is a prefix-code and (hence) uniquely decodable.
[08] Define a code c by E(1) = 00, E(2) = 01, E(3) = 10, E(4) = 11, E(5) = 100.(a) Show that the code sequence 00011011100 is uniquely decodable.(b) Show that the code sequence 100100 can be decoded into two different source sequences.
[32] Prove the following theorem. Let H(p1, p2,...,pn) be a function defined for any integer n and for all values p1, p2,...,pn such that pi ≥ 0 (i = 1, 2,...,n), n i=1 pi = 1. If for any n this function is continuous with respect to all its arguments, and if it has the properties in Items (a),
[26] Prove that the entropy function H has the following four properties:(a) For given n and n i=1 pi = 1, the function H(p1, p2,...,pn) takes its largest value for pi = 1/n (i = 1, 2,...,n). That is, the scheme with the most uncertainty is the one with equally likely outcomes.(b) H(X, Y ) = H(X)
[10] Let M be the set of possible source messages, and let E : M → {0, 1}∗ be a prefix-code.(a) Let M be a set of messages using symbols in an alphabet A. Show that if E is a prefix-code on A, then the homomorphism induced by E is a prefix-code on M.(b) Show that the Shannon–Fano code
[13] Suppose we have a random variable X that can assume valuesa, b,c, d with probabilities 1 2 , 1 4 , 1 8 , and 1 8 , respectively, with no dependency between the consecutive occurrences.(a) Show that the entropy H(X) is 7 4 (bits per symbol).(b) Show that the code E with E(a) = 0, E(b) = 10,
[10] To use an extra symbol like 2 is costly when expressed in bits. Show that the coding of strings consisting of k zeros and ones and one 2 requires messages of about k + log k bits.Comments. Hint: there are 2k(k + 1) such strings.
[M33] Consider a Bernoulli process with unknown probability p of a successful outcome. Assume that the prior probability of the bias p is uniformly distributed over the real interval (0, 1). Prove that after m successful outcomes in n independent trials, the expectation of a successful outcome in
[M42] Consider a sequence generated by a (p, 1 − p) Bernoulli process. Show that lim supn→∞ S∗n/√2 ln ln n = 1 with probability one.Comments. For reasons of symmetry, lim infn→∞ S∗n/√2 ln ln n = −1.This remarkable statement, known as the law of the iterated logarithm is due to
[M37] Consider one-way infinite binary sequences generated by a (p, 1 − p) Bernoulli process. Let Sn be as in Exercise 1.10.1.(a) Show that for every > 0, we have probability one that |pn−Sn| < n for all but finitely many n.(b) Define the reduced number of successes S∗n = (Sn −pn)/np(1
[M30] Prove the limit in Equation 1.8 associated with the condensation of the posterior probability.Comments. This may be called the inverse weak law of large numbers, since it shows that we can infer with great certainty the original probability (drawn from an unknown distribution) by performing a
[30] In an infinite sequence of outcomes generated by a (p, 1−p)Bernoulli process, let A1, A2,... be an infinite sequence of events each of which depends only on a finite number of trials in the sequence. Denote the probability of Ak occurring by Pk. (Ak may be the event that k consecutive 1s
[25] Let an experiment in which the outcomes are 0 or 1 with fixed probability p for outcome 1 and 1 − p for outcome 0 be repeated n times. Such an experiment consists of a sequence of Bernoulli trials generated by a (p, 1 − p) Bernoulli process, explained at the beginning of this section.Show
[O35] Investigate the problems as in Exercises 1.9.6 and 1.9.7, for push-down automata (PDA), time- or space-bounded classes, and primitive computable functions.
[33] Investigate related problems as in Exercise 1.9.6 by replacing finite-state machines (that is, regular languages) by slightly more complex languages such as deterministic one-counter languages or linear languages. Show that there are languages of both types such that selection according to
[32] What happens if we restrict our set of admissible selection functions to those computable by finite-state machines instead of Turing machines? First we need some definitions. Let ω = ω1ω2 ... be an infinite binary string. For each m = 0, 1, 00, 01,..., let fn be the number of occurrences of
[M40] Let ω1ω2 ... be an infinite binary sequence, and let fn =ω1 + ω2 + ··· + ωn.(a) A sequence ω is said to satisfy the infinite recurrence law if fn = 1 2n infinitely often. It can be shown that the set of infinite binary sequences having the infinite recurrence property has measure one
[10] Given an infinite binary string that is a collective with limiting frequency of 1s equal to p = 1 2 , show how to construct a collective over symbols 0, 1, and 2 (a ternary collective) with equal limiting frequencies of 1 3 for the number of occurrences of 0s, 1s, and 2s.Comments. Hint: use
[15] Suppose the sequence a1a2 ...ai ... has all the properties of a collective with p = 1 2 .(a) The limiting relative frequency of the subsequence 01 equals the limiting frequency of subsequence 11. Compute these limiting relative frequencies.(b) Form a new sequence b1b2 ...bi ... defined by bi =
[15] Suppose we are given a coin with an unknown bias: the probability of heads is p and of tails is (1 − p), p some unknown real number 0
[08] Consider the sequence 101001000100001000001 ..., that is, increasing subsequences of 0s separated by single 1s. What is the limiting relative frequency of 1s and 0s? Is this sequence a collective?
[13] A real number r ∈ [0, 1] is called a computable real number if there exists computable function φ such that r = 0.ω with φ(i) = ωi, for all i. We call ω a computable sequence. A computable sequence of computable reals is a sequence r1, r2,... of reals if there is a computable function
[37] In Exercise 1.7.16 we studied the notions of reducibility and degree. We now look at a coarser classification. An n-ary relation R is in the arithmetic hierarchy if it is either computable or, for some m, can be expressed as{x1,...,xn : (Q1y1)...(Qmym)S(x1,...,xn, y1,...,ym)}, (1.5)where each
[39] Let f be any computable function. Prove that there exists an n such that φn = φf(n).Comments. This result, and its elaborations to more complicated versions, is usually called the second recursion theorem or the fixed-point theorem for computability theory. The n is called a fixed-point
The function BB is defined in terms of Turing machines with a purely binary tape alphabet (no blanks) in quintuple format (rather than quadruple format used before). For each n define the set An = {i : Ti has n states and φi(0) < ∞}. That is, Ti with i in An halts when it is started with p = .
[35] Consider the generalized exponential function f informally having the following property: f(0, x, y) = y + x, f(1, x, y) = y × x, f(2, x, y) = yx, ... . A more formal definition of f is given by f(0, 0, y) =y, f(0, x + 1, y) = f(0, x, y) + 1, f(1, 0, y) = 0, f(z + 2, 0, y) = 1, f(z +1, x + 1,
[35] Use the definitions given earlier. The following is known as Post’s problem (1944). The computable sets have lower r-degree of unsolvability than K0, which has the highest r-degree of unsolvability in the computably enumerable sets. Are there other r-degrees of unsolvability (all r = 1, m, T
[35] In order of increasing generality we define some notions of reducibilities among sets:A set A is one-to-one reducible to a set B (A ≤1 B) if there exists a one-to-one computable function f such that for all x we have x ∈ A iff f(x) ∈ B.A set A is many-to-one reducible to a set B (A ≤m
[25] (a) Show that there exists an infinite set having no infinite computably enumerable subset. Such sets have been called immune by J.C.E. Dekker.(b) If a set with this property has a computably enumerable complement, then this complement was called simple by E.L. Post in 1944. Show that there
[20] A set A is computably enumerable without repetitions if A equals the range off, for some f that is computable and one-to-one.Prove that A is infinite and computably enumerable iff A is computably enumerable without repetitions.
[15] (a) Show that the function τ(x, y)=(x2 + 2xy +y2 + 3x+y)/2 is a computable one-to-one mapping from N 2 onto N . Show that this is not a prefix-code.(b) Show that τ (k) defined by τ (2) = τ and τ (k)(x1, x2,...,xk) = τ(τ (k−1)(x1, x2,...,xk−1), xk) is a computable one-to-one mapping
[20] Let K = {x : φx(x) < ∞}. Prove that K is a computably enumerable set that is not computable.
[34] (a) Show that it is possible to effectively enumerate the partial computable functions without repetition.(b) Let A = {x : φx is a total function}. Prove that A is not computably enumerable.Comments. Items (a) and (b) are not contradictory. Hint: In Item (a), dovetail the computations of all
[16] Prove that a set A is computably enumerable iff A is the range of some partial computable function iff A is the range of a total computable function or ∅.
[15] Prove that a set A is computably enumerable iff A is the domain of a partial computable function.Comments. This is often called the basic theorem of computably enumerable sets.
[20] Prove that the predicate f defined as f(x) = 1 if φx is total, and f(x) = 0 otherwise, is not total computable.
[20] Show that there is no computable function f such that f(x) = 1 if φx(x) is defined, and f(x) = 0 otherwise.Comments. This fact is known as the computable unsolvability of the halting problem.
[35] If P is the class of all partial computable functions (for convenience restricted to one variable), then any map π from N (the natural numbers) onto P is called a numbering. The standard indexing of the Turing machines provides such a numbering, say π0, and the indexing of the computable
[25] Show that for every m, n ≥ 1, there exists a computable function ψ = s(m+1) n of m + 1 variables such that for all x, y1,...,ym,φ(m+n) x (y1,...,ym, z1,...,zn) = φ(n)ψ(x,y1,...,ym)(z1,...,zn), for all variables z1,...,zn. (Hint: Prove the case m = n = 1. The proof is analogous for the
[11] It is important to distinguish between computable functions and the algorithms that compute those functions. To each function there correspond many different algorithms that compute it. Show that in the effective enumeration of Turing machines, as treated in this section, each partial
[15] Georg Cantor proved that the total functions are not countable by introducing his famous diagonalization argument. Suppose the contrary, and count the functions in order f1, f2,...,fi,... . Define a new function g, which we shall prove to be not in this list, by g(i) =fi(i) + 1, for all
[08] Show that there are countably infinitely many partial computable functions and also that many (total) computable functions.
[10] We can rework the effective enumeration of Turing machines and the definition of universal Turing machines using only tape alphabet A = {0, 1} (as opposed to {0, 1, B}) such that the d(A)d(Q) product increases at most by a fixed constant under the change of the original Turing machine to its
[15] The uniform distribution λ over the continuous sample space S = {0, 1}∞, the set of one-way infinite binary sequences (the half-open interval of real numbers [0, 1)), is described in Example 1.6.5.Let the Ln’s be as before. Show that λ(Γx) = Ll(x)(x), for all x = {0, 1}∗.
[M12] The uniform distribution over the countable sample space S = N can be defined as the probability density function L(x)=2−2l(x)−1 or alternatively, 6π2(l(x) + 1)2 2−l(x).(a) Show that in both cases x∈S L(x) = 1.(b) Let S1, S2,... be a sequence of sample spaces with Sn = {x : l(x)
[08] Which probability is greater: to score at least one ace (a 6)in six throws of a die, or to score at least two aces in 12 throws of a die?Comments. This question was submitted to Isaac Newton by the famous diarist Samuel Pepys in 1693. Newton answered that an easy computation shows that the
[10] Show that the probability of obtaining at least one ace (a 6)in four throws with one die is greater than the probability of obtaining at least one double ace in 24 throws with two dice.Comments. This is known as Chevalier de M´er´e’s paradox. It was posed by this passionate gambler to
[15] What is the probability that in a party of 23 people at least two people have a common birthday? Assume that birthdays are uniformly distributed over a year of 365 days and that the people at the party constitute a random sample. Use the formula derived earlier.Comments. Note that contrary to
[15] Consider the population of digits {0, 1,..., 9}. Use the formula you have found earlier to check that the probability that five consecutive random digits are all different is p = (10)5/105 = 0.3024.Comments. Source: [W. Feller, Ibid.].
[17] A random sample of size k is taken from a population of n elements. We draw the k elements one after the other and replace each drawn element in the population before drawing the next element. What is the probability of the event that in the sample no element occurs twice, that is, our sample
[HM45] Let π(n) denote the number of primes that do not exceed n. Show that(a) a crude approximation is π(n) ∼ n/ ln n;(b) a better approximation isπ(n) = n ln n+n(ln n)2 +2!n(ln n)3 +3!n(ln n)4 + O n(ln n)5.Comments. The displayed formulas are called prime number theorems.Source: [R. Graham,
[15] (a) Show that f(n) = Ω(g(n)) iff f(n) = o(g(n)).(b) Show that f(n) = Θ(g(n)) or f(n) = o(g(n)) iff f(n) = O(g(n)).(c) Show that f(n) = O(g(n)) iff f(n) = Ω(g(n)) and f(n) = Θ(g(n)).
[20] It is well known that n1/n → 1 for n → ∞.(a) Show that n1/n = eln n/n =1 + (ln n/n) +O((ln n/n)2).(b) Use (a) to show that limn→∞ n(n1/n − 1) = ln n.
[15] Denote the Hardy–Littlewood version of Ω we gave in the main text by ΩH and the Knuth version by ΩK.(a) Show that for function f defined by log log f(n) = log log n, we have f(n)=ΩH(n) but for no > 0 does f(n)=ΩK(n1/2+) hold.(b) Show that nonetheless, while f(n) = O(n), for no > 0
[M30] It is natural to wonder how large 100! is approximately, without carrying out the multiplications implied by the definition. Prove the approximation n! ∼ √2πn(n/e)n.Comments. This celebrated approximation of the factorial function was found by James Stirling [Methodus Differentialis
[08] Show that f(n) = o(g(n)) implies f(n) = O(g(n)), but not vice versa.
[10] Show that f(n) = O(f(n)); c · O(f(n)) = O(f(n)) if c is a constant; O(f(n) + O(f(n)) = O(f(n)); O(O(f(n))) = O(f(n));O(f(n))O(g(n)) = O(f(n)g(n)); O(f(n)g(n)) = f(n)O(g(n)).
[07] Show that x = o(x2); sin x = O(1); x−1/2 = o(1); x + x2 ∼x2, and n k=1 kn = O(n3).
[10] (a) Show that E(x)=¯x is a prefix-code.(b) Consider a variant of the ¯x code such that x = x1x2 ...xn is encoded as x11x21 ... 1xn−11xn0. Show that this is a prefix-code for the binary nonempty strings with l(¯x)=2l(x).(c) Consider x = x1x2 ...xn encoded as x1x1x2x2 ...
[10] Let E be as above. Define the mapping · : N ×N →N by x, y = E(x)E(y).(a) Show that · is a total one-to-one mapping and a prefix-code.(b) Show that we can extend this scheme to k-tuples (n1, n2,...,nk) of natural numbers to obtain a total one-to-one mapping from N ×N ×···×N into N
[10] Let E : N→{0, 1}∗ be a total one-to-one function whose range is prefix-free. E defines a prefix-code. Define the mapping · :N ×N →N by x, y = E(x)y.(a) Show that · is total and one-to-one.(b) Show that we can extend this scheme to k-tuples (n1, n2,...,nk) of natural numbers to obtain
[07] (a) Show that for x ∈ N we have l(x) = log(x + 1).(b) Give another code c(x) for the natural numbers x = 1, 2,... such that l(c(x)) = log x.Comments. Hint for Item (b): Use the correspondence(, 1),(0, 2),(1, 3),(00, 4),(01, 5),(10, 6),(11, 7),....
[03] If ¯xyz¯ = 10010111, what are x, y, z in decimal numbers?Comments. 1, 2, 6.
[14] Define the occupancy numbers for n balls distributed over k cells as a k-tuple of integers (n1, n2,...,nk) satisfying n1+n2+···+nk =n with ni ≥ 0 (1 ≤ i ≤ k). That is, the first cell contains n1 balls, the second cell n2 balls, and so on.(a) Show that there are n n1,...,nk placements
[15] (a) Prove that the number of ways n distinguishable balls can be placed in k numbered cells such that the first cell contains n1 balls, the second cell n2 balls, up to the kth cell contains nk balls with n1 + ··· + nk = n is nn1,...,nk= n!n1! ··· nk!.This number is called a multinomial
[M34] Prove the following identity, which is very useful in the sequel of this book: Up to a fixed additive constant we have log n k= k log n k + (n − k) log n n − k +1 2 log n k(n − k).
[08] Show that n k= (n)k k! and n k= n n−k.
[12] Consider a random distribution of k distinguishable balls in n cells, that is, each of the nk possible arrangements has probability n−k. Show that the probability Pi that a specified cell contains exactly i balls (0 ≤ i ≤ k) is given by Pi = k i(1/n)i(1 − 1/n)k−i.Comments. Source:
17. (a) Solve recursively in terms of π1, guess the answer, and prove by induction. Alternatively, read part
16. Bishops move diagonally. Starting at a corner, a bishop can reach 32 vertices, of which 14 have degree 7, 10 have degree 9, 6 have degree 11, and 2 have degree 13.Finally, 40.
14. Solve the equation π = π P to find πk = ρkπ0, where ρ = p/q. The chain is positive recurrent if and only if p < q. In order to distinguish between transience and null recurrence, consider a simple random walk on S at the times of leftward steps.
13. The quick way is to look for a solution of the detailed balance equations.
. Now, Wa(n) is the sum of a(n)independent, identically distributed random variables, and a(n)/n → 1/μ.12. 52 1 + 52 2 + · · · + 52 51 ≤ 52(1 + log 51). Use conditional probability and Markov’s inequality for the last part.
11. Use the fact that Un ≥ x if and only if Wa(n) ≤ n, where Wk is the time of the kth return to i , and a(n) =(n/μ) + x pnσ2/μ3
10. Check that Q and π are in detailed balance.
9. (a) 8, (b) 1, (c) 10. Use Theorem 12.57 and symmetry for the last part.
8. Either slog it out, or consider a collapsed chain with two states, 1 and 0 (representing‘not 1’). Now use the result of Problem 12.13.2.
7. Assume reversibility. By passing the π term along the product, we may see thatπi1 pi1,i2 pi2,i3 · · · pin ,i1 = pi2,i1 pi3,i2 · · · pi1,inπi1 . For the converse, sum over the intermediate j2, . . . , jn−1 to obtain p j1, jn (n − 1)p jn, j1 = pj1, jn pjn, j1(n − 1), and let n
6. Either solve the equation π = π P, or argue as follows. If the state is i jk, then i was the last book chosen and, of the books j and k, j was the last chosen. Therefore,πi j k = αiαj /(αj + αk ).
5. The Markov property holds by the lack-of-memory property of the geometric distribution(Problem 2.6.5). For the invariant distribution, look for a solution of the detailed balance equations (you may pick an easy equation to start).
4. The chain is irreducible and P0(T > n) = bn, where T is the first return time of 0.Recall Problem 2.6.6. The final answer is πj = bjP i bi .
3. In ‘physical’models of this type, one can start by looking for a solution to the detailed balance equations. In this case, we have πi =????N i2????2N N.
2. Recall Example 12.15. The chain is reversible in equilibrium when 0 < αβ < 1.
1. πi = 1/N for i ∈ S.
16. Condition on the length of the service time.22. (c) The mean total is 1 2 tE(Nt ) = 1 2λt2, by either calculation or symmetry.Chapter 12
15. p(t) = e−λt cosh λt = q(t). The time to the first change of state has the exponential distribution with parameter λ. Use independence for the last part.
Rewrite this in terms of characteristic functions and appeal to the continuity theorem for a watertight argument.
14. The following argument is not completely rigorous but is illuminating. Let t →∞in the formula for G(s, t) given in Problem 11.7.13 to obtain G(s, t) → exp[ρ(s − 1)]where ρ = θ/μ. This is the probability generating function of the Poisson distribution.
(b) The answer is 0 if a < 1 and a/(1 +a) if a ≥ 1.
(iii) X − Y has the bilateral exponential distribution.
(ii) min{X, Y} has the exponential distribution with parameter 2λ.
13. (a) (i) 1 − e−λX is uniformly distributed on (0, 1).

Showing 800 - 900 of 3340

First
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Last

Probability An Introduction 2nd Edition Geoffrey Grimmett, Dominic Welsh - Solutions

Step by Step Answers