Answer question 1 please

steady-state VCCIOI'S If P is a stochastic matrix, then a steady-state vector (or equilibrium vector) for P is a probability vector q such that Pq=q It can be shown that every stochastic matrix has a steady-state vector. In Example 3, q is a steady-state vector for P. .375 . _ 62 5] 1s a steady-state vector for the population migration matrix M in Example 1, because M _ .95 .03 .375 _ .35625+.01875 _ .375 _ I '1' .05 .97 .625 _ .01875+.60625 _ .625 'q EXAMPLE 4 The probability vector q = I: :ctor Spaces If the total population of the metropolitan region in Example 1 is 1 million, then q from Example 4 would correspond to having 375,000 persons in the city and 625,000 in the suburbs. At the end of one year, the migration out of the city would be (.05) (375,000) = 18,750 persons, and the migration into the city from the suburbs would be (.03) (625,000) = 18,750 persons. As a result, the population in the city would remain the same. Similarly, the suburban population would be stable. The next example shows how to nd a steady-state vector. THEOREM 18 If P is an n x n regular stochastic matrix, then P has a unique steady-state vector q.Further,ifxois anyinitial stateandxk+1= ka fork = 0,1,2,...,tbenthe Markov chain {xk} converges to q as k > 00. This theorem is proved in standard texts on Markov chains. The amazing part of the theorem is that the initial state has no effect on the long-term behavior of the Markov chain. You will see later (in Section 5 .2) why this is true for several stochastic matrices studied here. EXAM PLE 6 In Example 2, what percentage of the voters are likely to vote for the Republican candidate in some election many years fr0m now, assuming that the election Outcomes form a Markov chain? SOLUTION For computations by hand, the wrong approach is to pick some initial vector x0 and compute x1 , . . . , to; for some large value of k . You have no way of knowing how many vectors to compute, and you cannot be sure of the limiting values of the entries in xk. The correct approach is to compute the steady-state vector and then appeal to Theorem 18. Given P as in Example 2, form P I by subtracting 1 from each diagonal entry in P . Then row reduce the augmented matrix: .3 .1 .3 0 [(Pl') 0]: .2 .2 .3 0 .1 .1 .6 0 Recall from earlier work with decimals that the arithmetic is simplied by multiplying each row by 10.1 3 1 3 0 l 0 9/4 0 2 2 3 0 ~ 0 1 15/4 0 1 1 6 0 0 0 0 0 The general solution of(P 1);: = 015 in = 3x3, x2 = 145x3,and x3 is free.Choosing x3 = 4, we obtain a basis for the solution space whose entries are integers, and frorn this we easily nd the steady-state vector whose entries sum to 1: 9 9/28 .32 w: 15 , and :1: 15/23 e .54 4 4/28 .14 The entries in q describe the distribution of votes at an election to be held many years from now (assuming the stochastic matrix continues to describe the changes from one election to the next). Thus, eventually, about 54% of the vote will be for the Republican candidate. I 1 Warning: Don't multiply only P by 10. Instead, multiply the augmented matrix for equation {P 1)): = 0by10. EXAMPLE 5 Let P = :4 7. Find a steady-state vector for P. SOLUTION First, solve the equation PX = X. Px- x=0 Px- Ix =0 Recall from Section 1.4 that Ix = X. (P - I)X = 0 For P as above, P-1=[:4 3]-[6 0]-[4-3] To find all solutions of (P - I)x = 0, row reduce the augmented matrix: [4 3 0]~[ 8 8 8]~[6 304 8] Then x1 = x2 and x2 is free. The general solution is X2 [314]. Next, choose a simple basis for the solution space. One obvious choice is 3 / 4 but a better choice with no fractions is w : (corresponding to x2 = 4). Finally, find a probability vector in the set of all solutions of Px = x. This process is easy, since every solution is a multiple of the solution w above. Divide w by the sum of its entries and obtain 9 = 4/7] As a check, compute 6/10 Pq = 4/10 9/8 1971-12/70 +28/70=40/70 =9 The next theorem shows that what happened in Example 3 is typical of many stochastic matrices. We say that a stochastic matrix is regular if some matrix power Pk contains only strictly positive entries. For P in Example 3, .37 .26 .33 p2 .45 .70 .45 .18 .04 .22 Since every entry in P2 is strictly positive, P is a regular stochastic matrix. Also, we say that a sequence of vectors {Xk : k = 1, 2, ...} converges to a vector q as k - co if the entries in xk can be made as close as desired to the corresponding entries in q by taking k sufficiently large.Denition. A vector with nonnegative entries that add up to 1 is called a probability vector. A stochastic matrix is a square matrix whose columns are probability vectors. A Markov chain is a sequence of probability vectors :30, 551, :32, . . . together with a stochastic matrix P, such that #12131], 53.22.1351, 15.32.1352,... The vectors wk represent the state of your experiment /measurement after I: discrete time periods, hence we call them state vectors. Example 1. (pgs. 256-257) In a metropolitan area 60% of the population lives in the city and 40% lives in the suburbs. Every year 5% of the city's population moves to the suburbs, and 3% of the suburban population moves to the city. Our initial state here is given by the probability vector _. _ .600 _ .400 ' The stochastic matrix (or migration matrix in this case) is given by .95 .03 P _ [.05 .97] The rst column of the matrix tells us what happens to those who live in the city. Of the city dwellers 95% stay in the city and 5% move to the suburbs. The second column says that 3% of the peOple in the suburbs move to the city and 97% of people stay in the suburbs. After 1 year, the population distribution will be given by .95 .03 .600 _ .582 .05 .97 .400 _ .418 That is, 58.2% will live in the city, and 41.8% will live in the suburbs. After 2 years, the population distribution will be given by .95 .03 .582 _ .565 .05 .97 .418 _ .435 ' That is, 56.5% will live in the city, and 41.8% will live in the suburbs. Definition. (pg. 259) If P is a stochastic matrix, then a steady-state vector (or equilibrium vector) for P is a probability vector q such that Pq = q. It is a theorem in Markov Chain theory that there's another way to think of the steady-state vector, and that's as a limit as k -> co. That is, what happens long term? Theorem. (pg. 261) If P is an n x n regular stochastic matrix, then P has a unique steady-state vector q. Further, if To is any initial state and Ck+1 = PEk for k = 0, 1, 2, ..., then the Markov chain {ck} converges to q and k -+ co. Problem 1. In the book, they verify that .375 .625 is a steady-state vector for the population migration matrix from the example above. a. In the book, it is just given. Use the method of Example 5 on page 260 to find the steady-state vector for the matrix 1.95 .03 .05 .97 .95 .03]k b. Compute .05 .97 To for large values of k (try k = 40, 80, 120) where To is the initial state of 60% city / 40% suburb as seen above. c. Try part (b) again with the initial state TO = 33 67 and compare your answer to your answer in part (b). d. If the population remains at a constant 1 million, about how many people will live in the city after a bazillion years (euphemism for a long period of time)