Example of a Markov chain

This is an example of a problem which is similar in avor to Problem 57 of Section 1.5. Its an example of something called a Markov chain, in which the future state of a system is completely determined by its present state. Look through this example and see how it relates to Problem 57.

In Seattle, its either sunny or rainy. If its sunny today, the probability that it will be sunny tomorrow is 50%, leaving a 50% chance that it will be rainy tomorrow. On the other hand, if its rainy today, the probability is 10% that it will be sunny tomorrow, and 90% rainy. Let x1 be the probability that today is sunny, and x2 the probability that today is rainy. So the probability that is is sunny tomorrow is 0.5x1 + 0.1x2 . Similarly, the probability that it is rainy tomorrow is 0.5x1 + 0.9x2 . (Dont just take my word for it! Convince yourself This all has a very natural interpretation as a 2 2 matrix 0.5 P = 0.5 (2) (1)

that these two expressions make sense!) matrix multiplication problem. Let P be the 0.1 , 0.9

and let x be the 2-dimensional column vector with entries denoting the probability that today is either sunny or rainy: x1 x= . x2 Well call this x todays state vector. We immediately notice that Px = 0.5 0.5 0.1 0.9 x1 0.5x1 + 0.1x2 = x2 0.5x1 + 0.9x2

Compare this to expressions (1) and (2)! What we have is that P times todays state vector is equal to tomorrows state vector! Now lets use this information to interpret P 2 x. Notice that P 2 x = (P P )x = P (P x). As already mentioned, if x is todays state vector, then P x is tomorrows state vector. Hence, P (P x) is just P times tomorrows state vector. But P times a state vector gives the state vector for the next day. Hence, P times tomorrows state vector gives the state vector for one day after tomorrow. That is, P (P x) is the state vector for two days from today! More generally, one can see that for any positive integer k , if x is todays state vector, then P k x is the state vector k days from today. Lets see an example!

Question
If today is sunny in Seattle, what is the probability that it will be sunny in three days? What if today is rainy?

Answer
To answer both these questions, rst lets calculate P 3 : P3 = 0.5 0.1 0.5 0.9 0.5 0.5 0.1 0.9 0.5 0.5 0.1 0.22 0.156 = 0.9 0.78 0.844 1 (Why?). So in three days, the state 0

If today is sunny, the state vector for today is x = vector will be P 3x = 0.22 0.78 0.156 0.844

1 0.22 = . 0 0.78

This tells us that if today is sunny, in three days there is a 22% chance that it will be sunny and 78% chance rainy. 0 Similarly, if today is rainy, the state vector is y = . Since 1 P 3y = 0.156 , 0.844

we see that in three days there is a 15.6% chance that its sunny, and 84.4% chance rainy. For further study, it can be shown (in a mathematical meaningful way) that as k goes to innity, P k approaches the matrix 1/6 1/6 . 5/6 5/6 1/6 . To see how this is 5/6 meaningful, suppose that this steady state vector x represents todays state vector (so theres a 1/6 chance of sun, 5/6 chance of rain). Then tomorrows state vector is given by We say in this case that the steady state vector for this P is x = Px = 1/6 5/6 1/6 5/6 1/6 1/6 6/36 = =x = 5/6 30/36 5/6

In other words, tomorrow has the exact same probabilities of rain and sun as today has! We can extrapolate this to see that every day in the future has the same probability of being either sunny or rainy. Indeed, with a little work, this (in some sense) means that 1/6 of the days in Seattle can be expected to be sunny, and 5/6 can be expected to be rainy!