24. Wald’s equation can also be proved by using renewal reward processes. Let N be a stopping time for the sequence of independent and identically distributed random variables Xi , i  1.

(a) Let N1 = N. Argue that the sequence of random variables XN1+1,XN1+2, . . .

is independent of X1, . . . ,XN and has the same distribution as the original sequence Xi , i  1.

Now treat XN1+1,XN1+2, . . . as a new sequence, and define a stopping time N2 for this sequence that is defined exactly as N1 is on the original sequence.

(For instance, if N1 = min{n: Xn > 0}, then N2 = min{n: XN1+n > 0}.)

Similarly, define a stopping timeN3 on the sequenceXN1+N2+1,XN1+N2+2, . . .

that is identically defined on this sequence as N1 is on the original sequence, and so on.

(b) Is the reward process in whichXi is the reward earned during period i a renewal reward process? If so, what is the length of the successive cycles?

(c) Derive an expression for the average reward per unit time.

(d) Use the strong law of large numbers to derive a second expression for the average reward per unit time.

(e) Conclude Wald’s equation.