Example 4.4 (Thamformlng a Process into a Markov Chain) Suppose that whether ornotitrainstoday mereviomweatheremdinnsmnghthelmm (hiya. Specically, suppose that if it has rained for the past two days, then it will rain tomor- row with probability 0.7; if it rained today but not yesterday, then it will rain tomorrow with pmbability 0.5; if it rained yesterday but not today. then it will rain tomorrow with probability 0.4; if it has not rained in the past two days, then it will rain tomorrow with probability 0.2. Ifwe let the state at time n depend only on whether ornot it is raining at timen, then the preceding model is not a Markov chain (why not?). However, we can transfm'm this model into a Markov chain by saying that the state at any time is determined by the weather conditions during both that day and the plevious day. In other words. we cansaythattheprocessisin state 0 if it rained both today and yesterday. state 1 if it rained today but not yesterday, state 2 if it rained yesterday but not today, state 3 if it did not rain either yesterday or today. The preceding would then represent a four-state Markov chain having a transition probability matrix 0.? 0 0.3 0 0.5 0 0.5 0 0 0.4 0 0.6 0 0.2 0 0.8 P: You should carefully check the matrix P, and make sure you understand how it was obtained