Q4. Please help

4. In a discrete MC with transition matrix P, the hitting time (also known as rst passage time) from state i to state j is defined as the number of transitions made by the process in going from state i to state j. Let fig\") be the probability that the first passage time from state i to statej is equal to n, i.e., fig") :2 Pr{Xn =j,Xr j(r = 1,2, ...,n 1)|X0 = i}. a) Show that for n > 1: 00 b) Define the mean hitting time as mi}- :2 211:0 11 fig\") (note that for i = j you obtain the mean recurrence time as special case). Show that the mean hitting times satisfy a system of linear equations: mi] = 1 +Zkjpikmkji Vii]- c) Write the analogous formulas for a continuous MC with generator matrix