Problem 3. (30 points) Let fX(t); t 0g be a continuous-time stochastic process with

state space S = f1; 2; 3g. Every time the process visits state i, it stays there for an ex-

ponentially distributed amount of time with rate ri, where r1 = 1, r2 = 2, and r3 = 1,

independent of everything else. The transitions to other states take place according to the

following structure:

When the process is in state 1, it next visits state 2 with probability 0.5 and state 3

with probability 0.5 independent of its sojourn time in state 1.

When the process is in state 2, it next visits state 1 with probability 0.5 and state 3

with probability 0.5 independent of its sojourn time in state 2.

When the process is in state 3, it next visits state 1 with probability 0.5 and state 2

with probability 0.5 independent of its sojourn time in state 3.

(a) Is fX(t); t 0g a CTMC? Why or why not? If it is a CTMC provide its rate matrix or

diagram.

(b) If the process starts in state 1 at time 0, what is the expected amount of time it takes

to visit state 3?

(c) If the process starts in state 1, explain how you would obtain the expected amount of

time the system spends in state 1 during the rst T units of time. (In your explanation, be

specic about the uniformization constant and the associated transition probability matrix

for the uniformized chain.)