Problem 3. (30 points) Let {X(t), t 0} be a continuous-time stochastic process with state space S = {1, 2, 3}. Every time the process visits state i, it stays there for an exponentially distributed amount of time with rate ri , where r1 = 1, r2 = 2, and r3 = 1, independent of everything else. The transitions to other states take place according to the following structure:

When the process is in state 1, it next visits state 2 with probability 0.5 and state 3 with probability 0.5 independent of its sojourn time in state 1.

When the process is in state 2, it next visits state 1 with probability 0.5 and state 3 with probability 0.5 independent of its sojourn time in state 2.

When the process is in state 3, it next visits state 1 with probability 0.5 and state 2 with probability 0.5 independent of its sojourn time in state 3.

(a) Is {X(t), t 0} a CTMC? Why or why not? If it is a CTMC provide its rate matrix or diagram.

(b) If the process starts in state 1 at time 0, what is the expected amount of time it takes to visit state 3?

(c) If the process starts in state 1, explain how you would obtain the expected amount of time the system spends in state 1 during the first T units of time. (In your explanation, be specific about the uniformization constant and the associated transition probability matrix for the uniformized chain.)

Problem 3. (30 points) Let {X (t),t 2 0} be a continuous-time stochastic process with state space 8 = {1,2,3}. Every time the process visits state i, it stays there for an ex- ponentially distributed amount of time with rate 1%, where r1 = 1, r2 = 2, and r3 = 1, independent of everything else. The transitions to other states take place according to the following structure: o When the process is in state 1, it next visits state 2 with probability 0.5 and state 3 with probability 0.5 independent of its sojourn time in state 1. 0 When the process is in state 2, it next visits state 1 with probability 0.5 and state 3 with probability 0.5 independent of its sojourn time in state 2. a When the process is in state 3, it next visits state 1 with probability 0.5 and state 2 with probability 0.5 independent of its sojourn time in state 3. (a) Is {X (t), t 2 0} a CTMC? Why or why not? If it is a CTMC provide its rate matrix or diagram. (b) If the process starts in state 1 at time 0, what is the expected amount of time it takes to visit state 3? (c) If the process starts in state 1, explain how you would obtain the expected amount of time the system spends in state 1 during the rst T units of time. (In your explanation, be specic about the uniformization constant and the associated transition probability matrix for the uniformized chain.)