ISE-230 Fall 2015 - Prof. B. Defourny HW 8 - Discrete Time Markov Chains Q1. Consider the following (one-step) transition matrix of a (discrete-time) Markov Chain, where the states are labeled from 0 to 4, 0 1/5 0 4/5 1/4 0 1/2 1/4 = 0 1/2 0 2/5 0 0 0 1 [1/3 0 1/3 1/3 0 0 1/10 . 0 0 ] a) Determine the communication classes of the Markov Chain b) For each class, determine if it is : transient | recurrent | periodic (indicate period) c) Say if the Markov Chain is: irreducible | ergodic. d) Suppose the chain is in state 1 at time t=2. Given that, determine the probability of not being in state 3 at time t=7. Q2. The weather on day t can be Sunny (0) or Rainy (1). If the weather yesterday was the same as the weather the day before yesterday, there is a 90% chance that the weather tomorrow is the same as the weather today. If the weather yesterday and the weather the day before yesterday differed, there is a 60% chance that tomorrow's weather differs from today's weather. You are asked to show that this problem can be formulated as a Markov Chain: a) Identify the states (be clear in your ordering of the states). b) Establish the (one-step) state transition matrix. c) Compute the steady-state probabilities