Python: Our goal is to perform an analysis of long-term (or steady-state) performance of a Markov Chain. Suppose you are given a transition matrix P. Your task is to estimate the long-run proportionsby observation. Of cause, you can achieve the same resultusing analytical methods. Thepurpose here is to illustrate the principle, so that you can then apply it to more complex systems, where analytical methods are intractable. The direct approach to obtain long-run performance prole is to perform a simulation, i.e., trace the transitionsoftheMarkovchainforacertainnumberofiterations,untilthelong-runbehaviorhasbeenestablished, counting the proportion of time that the system spends in each state. According to the theoretical result, if this simulation is performed for a long enough time, these proportions will converge to the long-run probabilities that you can nd by solving the system of equations as discussed in class. In your assignment you are asked to rst, perform a simulation estimating these probabilities, and then analyze the number of transitions that isrequiredforyoutoachieveareasonableaccuracy. Youcanestimate thelong-runproportionsby tracking the state of the system over a long period of time and counting the number of times that the system enters each state.

Directions:

As the underlying model consider Bonus-Malus insurance policy described in class and in your book (example 4.7). Each of you should use their own, individual value of the Poisson parameter . To obtain your , follow the link http://www.wolframalpha.com/input/?i=random+ number+between+0.1+and+2.0 using the value in the Result led.

Dene transition matrix P. For example, in Python a 33 matrix can be dened by: p = [[.4 , .4 , .2] , [.0 , .5 , .5] , [.0 , .4 , .6]] Here the rst row is (0.4,0.4,0.2), second row is (0,0.5,0.5), etc. In your case, you will need to dene a 44 matrix. To obtain the specic values for matrix elements, you can either precompute the transition probabilities according to the Poisson distribution or do it in Python according to the Poisson formula (the latter is the preferred method), see example 4.7 in the book and class discussion for details. Note that in Python lambda is a reserved word, so you will need to use a different name for the corresponding variable.

Dene an array of state counters (I will call it scnt); the number of simulation steps that you will use (I will call it nsteps); and the current state (state). Originally set scnt = [0 , 0, 0, 0] nsteps = 100 state = 0 The array will be used to count the amount of time that the systems spends in each state, if observed for nsteps iterations. Variable state keeps track of the current state.

Now, perform transitions nsteps times according to the transition matrix, recording the progression, i.e., repeat the following nsteps times.

increment the state counter that corresponds to the current state, i.e., record the current location

perform a transition. The next state is determined by the current state (state) and the corresponding row of the transition matrix. There are different ways to do that, one of which is to use a uniform random variable.

draw a realization of a uniform random variable (function u = random.random()) in Python

for the row of the transition matrix that corresponds to the current state, add the transition probabilities one by one from left to right until the sum exceeds the value of u. The rst time when that happens determines the next state

Compute the long run proportions by dividing each scnt by nsteps.

Analyze howthe values you obtain change withthe value of nsteps. Identify the valueof nsteps that is enough to determine the long-run proportions with at least two digits of accuracy.