5 Consider the success run chain in Example 8.2.16. Suppose that the chain has been

running for a while and is currently in state 10. (a) What is the expected number of

steps until the chain is back at state 10? (b) What is the expected number of times the

chain visits state 9 before it is back at 10?

16 Consider a version of the success run chain in Example 8.2.16 where we disregard

sequences of consecutive tails, in the sense that for example T, T T, T T T, and so on,

all simply count as T. Describe this as a Markov chain and examine it in terms of

irreducibility, recurrence, and periodicity. Find the stationary distribution and compare

with Example 8.2.16. Is it the limit distribution?

17 Reversibility. Consider an ergodic Markov chain, observed at a late timepoint n. If

we look at the chain backward, we have the backward transition probability qij =

P(Xn?1 = j|Xn = i). (a) Express qij in terms of the forward transition probabilities

and the stationary distribution ?. (b)If the forward and backward transition probabilities

are equal, the chain is called reversible. Show that this occurs if and only if ?ipij =

?jpji for all states i, j (this identity is usually taken as the definition of reversibility).

(c) Show that if a probability distribution ? satisfies the equation ?ipij = ?jpji for all

i, j, then ? is stationary.

18 The intuition behind reversibility is that if we are given a sequence of consecutive states

under stationary conditions, there is no way to decide whether the states are given in

forward or backward time. Consider the ON/OFF system in Example 8.2.4; use the

definition in the previous problem to show that it is reversible and explain intuitively.

19 For which values of p is the following matrix the transition matrix of a reversible Markov

chain? Explain intuitively.

P =

0 p 1 ? p

1 ? p 0 p

p 1 ? p 0

!

20 Ehrenfest model of diffusion. Consider two containers containing a total of N gas

molecules, connected by a narrow aperture. Each time unit, one of the N molecules is

chosen at random to pass through the aperture from one container to the other. Let Xn

be the number of molecules in the first container. (a) Find the transition probabilities

for the Markov chain {Xn}. (b) Argue intuitively why the chain is reversible and why

the stationary distribution is a certain binomial distribution. Then use Problem 17 to

show that it is indeed the stationary distribution. (c) Is the stationary distribution also

the limit distribution?

21 Consider an irreducible and positive recurrent Markov chain with stationary distribution

? and let g : S ? R be a real-valued function on the state space. It can be shown that

1

n

Xn

k=1

g(Xk)

P?

X

j?S

g(j)?j

for any initial distribution, where we recall convergence in probability from Section 4.2.

This result is reminiscent of the law of large numbers, but the summands are not i.i.d.

We have mentioned that the interpretation of the stationary distribution is the long-term

proportion of time spent in each state. Show how a particular choice of the function g

above gives this interpretation (note that we do not assume aperiodicity.

2. [1D mark{s}] You invite your friend to come to your house for dinner. They have to drive from a fair distance away, so the probability distribution of their arrival time t is uniform between 7pm and 3pm, assuming everything is going to plan. At '?:3pm, you start to worry because your friend has not arrived. Let Ill be the statement that your friend did not arrive before T:3[lpm. Let X be the statement that everything is fine1 so -IX is the statement that something is wrong. {a} [2 mark(s}] Find the probabilities P[l I X} and P[1 | :X] lets discussed in lectures, these relative frequencies or proportiors are not the same concept as the Bayesian probabilities needed for the question. but in some applications it is sensible to set them to the same values. {b} [2 marklsjl Using a Bayes Box or Bayes' rule, calculate the posterior probabilities of X and -X given DI. Use prior probabilities of 0.99 and (1.131 respectively. At T:54pm, your friend still hasn't arrived. Let D2 be the statement that your friend did not arrive before 7:54pm. {c} [3 mark(s]] Using a Bayes Box or anes' rule, calculate the posterior probabilities of X and :X given D1 and D2. Use the method where you start from the original prior probabilities and musider the data jointly. lid} [3 mark(s}] Using a Bayes Box or Bayes' rule, calculate the posterior probabilities of X and -IX given DI and D2. Use the method where your prior is the posterior from part (b) and the updating process only uses D2. Hint: You should get the same result in parts {c} and {d}. 2 Concepts 3. From Table 3, select three (3) and for each, briefly explain the underlying concept in your own words and using the appropriate vocabulary terms. Also briefly provide a real-world example/application. (20 points total) Frequentist probability Classical probability Subjective probability Addition Rule for probability Multiplication Rule for probability Independence of events Bayes' Rule Factorial Principle Permutation Principle Combination Principle Multinomial Principle Expected value Binomial distribution Poisson distribution Normal distribution Standardization Sampling distribution Central Limit Theorem Normality of sample data Normal approximation to Binomial Table 3: Concepts for Question 3Dr. Rooker is studying spotted seatrout and is interested if the size of seatrout in Galveston Bay is different from the seatrout in Matagorda Bay. a. What are Dr. Rooker's null and alternative hypotheses? b. Using the information below, determine if there is a significant difference in seatrout size in Galveston Bay and Matagorda Bay at a = 0.05? Galveston Bay: mean = 39 cm + 10 SD; n = 50 Matagorda Bay: mean = 35 cm + 8 SD; n = 30For the following pairs of assertions, indicate which do not comply with our rules for setting up hypotheses and why (the subscripts 1 and 2 differentiate between quantities for two different populations or samples): 1) HO: p1 - p2 = -0.1, Ha: p1 - p2