Suppose that whether or not it rains today depends on previous weather conditions

through the last two days. Specifically, suppose that if it has rained for the past two days, then it

will rain tomorrow with probability 0.7; if it rained today but not yesterday, then it will rain

tomorrow with probability 0.5; if it rained yesterday but not today, then it will rain tomorrow with

probability 0.4; if it has not rained in the past two days, then it will rain tomorrow with

probability 0.2.

If we let the state at time n depend only on whether or not it is raining at time n, then the above

model is not a Markov chain. Why? Because the Markov property requires that what the weather

is tomorrow depends only on the weather today, but in this example, the weather tomorrow

depends on the today's and yesterday's weather. However, we can transform the above model

into a Markov chain by saying that the state at any time is determined by the weather conditions

during both that day and the previous day. In other words, we can say that the process is in

state 0 if it rained both today and yesterday,

state 1 if it rained today and but not yesterday,

state 2 if it rained yesterday but not today,

state 3 if it did not rain either yesterday or today.

The preceding would then represent a four-state Markov chain having the following transition