IEOR 161: EXAM 2

APRIL 16, 2015

(1) (2 pts each) Indicate whether each of the following statements about Markov chains is true or false.
If false provide a counter example. If true briefly explain why.
(a) If the expected time to return to state i is infinite, then i is a transient state.
(b) If {Xn }
n=0 is a Markov chain, then X3 is independent of X1 .
(c) Every Markov chain has a stationary distribution.
(d) The limiting distribution of a Markov chain is always independent of the starting distribution.
(e) If i is recurrent, then the limiting probability of being in state i is always positive.
(2) (6 pts) Find all stationary distributions for a Markov chain whose transition Probabilities are given
as follows:
1
1
P01 = P03 = , P12 = 1, P21 = P22 = , P34 = 1, P43 = 1.
2
2
(3) (2 pts each) Suppose die A is 6-sided and die B is 5-sided. (That is die A has faces numbered 1
through 6, die B has faces numbered 1 through 5, and for each die, each face is equally likely to
appear). For the first roll we choose either die A or B with probability 12 . If the outcome of any
roll is even, we roll with the same die for the next time, while if the outcome of the roll is odd, we
switch to the other die for the next time. We continue rolling indefinitely following this process.
(a) What is the probability both die have been rolled after 2 rolls?
(b) What is the probability die B is rolled on the second roll?
(c) What is the expected value of the number rolled on the third roll? (You dont need to simplify
the answer.)
(d) What is the long term fraction of rolls that use dice A?
(e) What is the expected number of rolls required to first roll a 1?
(4) (2 pts each) Consider the following model for the price of a stock. Each day the price of the stock
either goes up by $1 with probability p or goes down by $1 with probability q = 1 p. Assume these
price movements are independent of the current and past stock prices. Suppose that on Monday you
buy the stock for $5, and you decide to sell when the price either falls to $4 or rises to $7.
(a) What is the probability you sell for $7 on Friday (after 4 price changes)?
(b) What is the probability you sell on Friday (for either $4 or $7)?
(c) What is the expected number of days you hold the stock?
(d) While you hold the stock, what is the expected number of days the price is $6?
(e) What is the probability you make a profit?
(f) What is your expected selling price?