Stochastic modelling;

thlm'l Let :1 E [I], 1] and c = (1 Lilljd. A Marker chain has transition matrix 1 2 3 at 5 6 I] d c c c c I] I] 1 I] .1 .2 .3 .4 I] I] 2 I] .2 .3 .4 .1 I] I] P = 3 I] .3 .4 .1 .2 I] I] 4 I] .4 .l .2 .3 I] I] 5 I] c c c c I] d 6 I] c c c c d U {a}. Determine, for each value ef ti E [I], 1] the pnsitive recurrent state{s','l [if any}, and the null recurrent stata [if any}. {h} Determine, for each value {if d E [I], 1], the cennnunicaticn classes. {c}. Fer what if an},r value{s} cf at will the Marker chain be regular? {:1}. Let r = (mg, #1,. .. ,11'5] = [I], %, dl,%, %, I],I]]. For what if any,r value[s) cf d will 11' he i} a. stationary distributien'? ii} a. limiting distribution? {e}. Assume that the Marker chain starts in state I]. Determine, for each value cf [1' E [I], 1] fer which state It] is transient, the expected time until the Marker chain leaves the communication class state I] helnngs tn. Problem 2 Throughout this problem, Y will be a birth/death-process with state space {0. 1, 2}. The intensity of an upward jump from state i ( {0, 1} will be 2 - i. The inten- sity of a downward jump from state ic {1, 2} will be }2. (a). Write down i) the generator, ii) the Kolmogorov backward equation (i.e. equation system) and iii) the Kolmogorov forward equation (i.e. equation system) associated to Y. (b). In the long run, what fraction of the time will Y spend in state 1?Problem 3 Throughout this problem, M(t) will be a martingale in continuous time, starting at M(0) = 0, + will be first time for which (M| 2 1, and M satisfies E[(M()) ] dB(t). X(0) = x (1) where A c Rox and E e Roxm are matrices with constant entries, b(t) is a deterministic function taking values in R", and B is m-dimensional standard Brownian motion. (a). Solve equation (1) (i.e.: write X as a function of x, an integral wrt. t and an integral wrt. the Brownian motion, but do not try to evaluate all the integrals explicitely). (b). Simplify as much as possible if b(t) = Aq, where q does not depend on t. In parts (c) and (d), assume that X satisfies (1), with b = Aq as in (b), and A = al + BC. (c&d) where C has the property that C- = C. Here, a, S and y are constant numbers. (c). Show that (2) (d). Use (2) to simplify the solution of (1) as much as possible. You are allowed to use equation (2) regardless of whether you managed to prove it. (Hint: you can use without proof the fact that ext = eres is true whenever the matrices commute, i.e. whenever KL = LK.)Now drop the condition, and consider instead for the rest of the problem, the system of stochastic differential equations for X = (X1, X2, X3) : dXi(t) = m(X2(t) - Xi(t))dt + o, dB(t) dX2(t) = me(X3(t) - X2(t))dt +0, dB(t) (3) dX3(t) = ra(m - Xa(t))dt + 03 dB(t). where all three r; are > 0, and where X starts at X(0) = x. (e). Write equation system (3) on the form (1), and show that when all the r, are distinct (i.e. mi # re # ry # mi), then A has eigenvectors (right eigenvectors, not left!) u1 = (1, 0, 0) , u2 = (1, 1-72. 0) , 13 = (1, 1-2, (1-5)(1-2)). (f). Let U be the matrix with columns up, us and us. Find deterministic real-valued func- tions di(t), de(t), de(t) such that whenever all the r; are distinct, we have di(t) 0 PA! = UA(t)U-1, where A (t) = 0 d2 (t) 0 (4) 0 0 da(t) (You are not supposed to calculate U-1.) (g). Assume that E is the identity matrix. Use your diagonalisation (4) to simplify the solution X of (3). Your formula should not include A explicitely. (Again, you are not supposed to calculate U-1.) (h). Again, consider equation system (3) written on the form (1), but assume now that 71 = 12 = 13. Is the matrix A now diagonalisable