MARKOV CHAINS MODEL

Mustufa H. Abidi

Introduction

Markov Property: The state of the system at time t+1 depends only on the
state of the system at time t

Pr X t 1 xt 1 | X 1 X t x1 xt
X1

X2

X3

Pr X t 1 xt 1 | X t xt
X4

X5

Example

Suppose that an Orange Juice (OJ) company controls 20% of the OJ market.
They hire a market research company to predict the effect of an aggressive
ad campaign. It was estimated that:
- Someone using Brand A will stay with Brand A with a probability of 90%
- Someone not using Brand A will switch to Brand A with a probability of 70%

Let us suppose that families buy OJ once a week.

A = Brand A users

A = Other than A brand users

Transition Diagram
0.1
0.9

A
0.7

0.3

Transition Matrix

P=

A
A
Current State

Next State

State Distribution Matrix

Initial

State Distribution Matrix

So =

0.2

0.9

0.1

0.7

0.3

Initial State
0.8
A

Probability to Use Brand A

Probability

of Using Brand A after one week = So * P

= [0.74 0.26] = S1 (First state matrix)

Therefore after 2 weeks, S2 = S1 * P
= [0.74 0.26] *

Similarly,
Hence,

S2 = [0.848 0.152]
S3 = [0.8696 0.1304]
Sn = Sn-1 * P

One more Example

A new rapid transit system (RTS) has just started in a city. In the first month
of operation, it was found that 25% of commuters are using the RTS, while
75% were still travelling with other modes (OM). The following transition
matrix was determined from records of other rapid transition systems:

Transition Matrix (P) =

The study was conducted to answer three questions:

1. What is the Initial-State Matrix?

2. What % of the commuters will be using the RTS after 1 st month and 2nd
month?
3. Find the % of commuters using each type of transportation after it has
been in service for a long time?

Example (RTS)

So =
S1 = So * P =

S1 = [0.425 0.575]
Similarly,
S1 = [0.5125

0.4875]

Example (RTS)

Stationary Matrix (S) =

S*P=S
[SRTS

SOM] *

= [SRTS

SOM]

SRTS + SOM = 1
By solving above system of equations, we get:
S = [0.6 0.4]
This mean after long run 60% of people will use RTS.