Parrondo’s Paradox, Brownian Ratchets

and Game Theory

CS 541 Term Project

Bruce Graham
12/19/2002

Electronic Copy: http://www-personal.engin.umich.edu/~bftsplyk/CS541/Parrondo

Bruce Graham’s: E-mail

Bruce Graham 1 4/26/07

Overview

The word paradox comes to us from Greek and Latin and means “contrary to
expectation or opinion”. Many of the systems studied in this course had this particular
feature. Something about them was “contrary to expectation”. The particular paradox of
interest had its origin just about forty years ago when Feynman [1963] discussed a
ratchet and pawl, whose movement depended on converting collisions of Brownian
particles with a paddlewheel into work apparently in violation of the second law of
thermodynamics. The apparent violation was resolved by allowing the apparently
irreversible machine to be reversible. This example was qualitatively persuasive, but
apparently not rigorous. Several people have tried to duplicate the analysis with varying
degrees of success.

Brownian motion, is one of the effects of molecular motion, and is often used to describe
stochastic processes. A stochastic trajectory is characterized by its mean and its
variance. Unlike chaotic systems, which have a sensitive dependence on initial
conditions, stochastic processes would seem to have no dependence on initial
conditions for two reasons. Starting the system over at the same initial condition cannot
reproduce a stochastic trajectory. Every run from the identical initial condition produces
a different result. The second reason is that stochastic process with the Markov
property, which is a large class of processes, depends only on their present state and
not on their history. That is they have no memory. If S0 is the initial condition, and S1 is
the first successor no future state can be affected by the initial conditions. The Flashing
Brownian Ratchet has a resemblance to the experiment described by Feynman in which
a supposedly irreversible machine is shown to allow motion in either direction. The
surprise is that it favors motion in the direction, which would be “contrary to expectation”
and thus arises the paradox.

The last piece of this tale was the challenge to create a discrete version of the Flashing
Brownian Ratchet. Professor Juan M.R. Parrondo, a Professor of Physics at the
Universidad Complutense de Madrid, created a pair of losing games which when played
alternately produced consistently positive gains. This then is Parrondo’s paradox, which
we will describe and attempt to explain.

Bruce Graham 2 4/26/07

The Flashing Brownian Ratchet

A ratchet is part of a mechanical device called a ratchet and pawl which only allows
motion in a single direction. An example of this machine can be seen on a tennis court
where a crank turns a toothed wheel, called the ratchet, on a shaft to apply tension to a
cable holding up the net. When the torque on the crank is released, the shaft may go in
reverse a bit, until the pawl drops into the valley between the teeth and prevents the
ratchet from rotating in the reverse direction.

H ig h
P o te n tia l

B ro w n ia n P a rtic le

R a tc h e t “O N ”

R a tc h e t “O F F ”

Low
P o te n tia l

F ra n z -J o s e p h E lm e rs ’s W e b s ite

h ttp ://m o n e t.p h y s ik .u n ib a s /c h /~ e lm e r/b m /in d e x .h tm l

Figure 1 – Flashing Brownian Ratchet

Brownian motion, an effect of molecular motion, is the random motion of a group of

particles. This Brownian motion has mean zero and variance σ2. Geometric Brownian
motion, the underlying basis for the Black-Scholes Partial Differential Equation, which is
a member of the family of Parabolic Partial differential Equations, has a non-zero mean
or drift and a variance σ2. Figure 1 is a crude drawing of a Java simulation on Franz-
Joseph Elmer’s website which exhibits the behavior of the Flashing Brownian Ratchet.
For the Flashing Ratchet we assume the mean zero Brownian motion and that the
particles are subject to a potential. This potential has a stationary component and a
time-varying component. To create the Flashing Ratchet the time varying component of
the potential is switched “ON” and “OFF” at a periodic rate. When the ratchet is “OFF”
the Brownian Particle represented by the ball tends to roll downhill from the region of
“High Potential” to the region of “Low Potential”. When the ratchet is switched “ON”
depending on the position and velocity of the particle it may roll into a valley on the left or
a valley on the right of its current position. The claim is – “if it rolls to the left enough
times – it will march uphill”. This is “contrary to expectation”, but is supported by the
mathematics, and the simulation.

Bruce Graham 3 4/26/07

Game Theory

The development Game Theory is described in the seminal work by von Neuman and
Morgenstern[1944], which they fully expected to be useful in designing better economic
and social policies. Their work was popularized by Williams[1954] under contract from
the RAND Corporation. The game of interest at that time was “GLOBAL
THERMONUCLEAR WAR”. The essential question was – could you win one? That we
are still here probably means that the answer was no. Some other titles in the series
were “Air War and Emotional Stress” and “Soviet Attitudes toward Authority”.

The elements of Game Theory are easy to explain. Each game has two or more
players. These players need not be persons; rather we count sets of opposing interests.
An example is a five-handed game of stud poker where two players have formed a
coalition, and thus the game is now a four-player game. Each player has two or more
choices, which can be selected in each round of the game. After each player makes his
respective choice, a payoff is revealed according to the rules of the game. The payoff of
a two player game is represented with an ordinary matrix. The two players, called A and
B, have an mxn matrix M. Player A is assigned to the rows of M, and player B is
assigned to the columns of M. Then A’s choice is an integer row number “i” in the range
[1..m], and B’s choice is a column number “j” in the range [1..n]. The payoff for these
choices on this round is just the element of M at row i and column j, or Mij. By
convention a positive number means that player A “receives” the payoff from player B,
and a negative number means that player B “receives” the payoff from player A. Play
continues with another round if possible and desirable. This is the description of a zero
sum game. This means that A’s winnings are equal to B’s losses. A non-zero sum
game can be found in the stud poker game where by agreement the winner might have
to contribute 10% of his winnings for drinks and refreshments. There might also be a
member of law enforcement who requires a gratuity to ignore gambling on his beat.

The way in which players make their choices over multiple rounds of the game is called
a strategy. Williams[1954] defines a strategy as “a complete plan that cannot be upset”.
The sense of this definition is that a player who can find, and stick to his optimal strategy
cannot be at a disadvantage. He may still loose, but his losses will be minimized. A
player who cannot find, or does not know, or deviates from his optimal strategy will give
his opponent an advantage he would not normally enjoy. In some cases a player will
have only a single optimal choice on each round, this is called a pure strategy. When
such a strategy exists, any deviation from it will result in larger than expected losses.
When both players have a pure strategy the game is said to have a saddle point.
Games with saddle points sound trivial and boring, except that our example of Global
Thermonuclear War may have been such a game. In other cases a player will alternate
his choices using some random device such as cards, coins, dice or a random number
generator chosen to give the proper distribution. This is called a mixed strategy.
Example – The River Tale

Bruce Graham 4 4/26/07

This example is taken from Williams[1954]. “Steve is approached by a stranger who
suggests they match coins. Steve says that it is too hot for violent exercise. The
stranger says, “Well then, lets just lie here and speak the words ‘heads’ or ‘tails’—and to
make it interesting I’ll give you $30 when I call ‘tails’ and you call ‘heads’, and $10 when
it’s the other way around. And—just to make it fair—you give me $20 when we match.
Warned by the environment (they are on a Mississippi Riverboat) Steve suspects he
should have the man arrested, rather than play with him. The average loss to Steve is
$1.25 and his optimal strategy is to call ‘heads’ and ‘tails’ in a 3:5 ratio.”

The River Tale Stranger

H T
H ($20.00) $30.00
Steve
T $10.00 ($20.00)

Table 1 – Payoff Matrix

Even if Steve follows his optimal strategy his expected loss per game is $1.25. To see
how these numbers are obtained for Steve’s choices we subtract an element of column 1
from an element of column 2, and write the absolute value of the result next to the
opposite row.

Row 1 : 10 − (− 20 ) = 30
Row 2 : (− 20) − 30 = 50

This tells Steve that he must use a mixed strategy calling ‘heads’ and ‘tails’ in the 3:5
ratio. Similarly, the Stranger’s optimal strategy, which he presumably already knew is to
call ‘heads’ and ‘tails’ in a 5:3 ratio. The expected value of the game is just

E [game] =
(30 ⋅ $30.00) + (50 ⋅ −$20.00 ) = −$1.25
(30 + 50)

Bruce Graham 5 4/26/07

Parrondo’s Games

Prior to inventing the paradoxical games Parrondo published in 1996 a paper which
criticized Feynman’s analysis of the Brownian ratchet. Considering Feynman’s iconic
status in the field of Physics, this took some courage. The paradoxical game was the
result of a challenge to create a discrete analog to the Brownian Ratchet. The games
are described in a paper submitted in 2002 and also on his website.

To keep track of the progress of our games we presume that Player A has some amount
of capital with which to play the games. If Player A has an amount of capital called X(t)
and if <X(t)> is the average amount of capital, then a game is winning if <X(t)> is a
monotonically increasing function of t. A game is fair if <X(t)> is constant. A game is
losing if <X(t)> is a monotonically decreasing function of t. Player B has no role in this
game except as a bank for Player A’s wins or loses

Parrondo describes two games, A and B. Each game has a negative payoff or expected
value. Game A can is modeled by Player A flipping a slightly unfair coin while Player B
does nothing. The payoff for Player A in Game A is $1.00 from Player B for a head and -
$1.00 to Player B for a tail. The P(heads) is just a bit less then ½, while the P(tails) is
just a bit more than ½. Game A is a two player zero sum game, but neither player
chooses a strategy. The strategy for both players is effectively chosen by the unfair
coin. In several papers, web pages, and simulations a “bit more” or a “bit less” is taken
as 0.005. With these probabilities we can calculate the expected value of Game A as
follows:

P (head ) = − 0.005 = 0.495, P (tail ) = + 0.005 = 0.505

1 1
2 2
E [Game A] = (0.495 ⋅ $1.00 ) + (0.505 ⋅ −$1.00 ) = −$0.01

Thus we can see that Game A is a losing game.

Game B is more complicated to describe and to analyze. In Game B there are two
coins. Both coins are unfair, but in different directions and by different amounts. To
keep these coins separate from the coin in Game A we label that “coin #1”. The Game B
coins will be label “coin #2” and “coin #3”. As in Game A, Player A will flip one of the two
coins and win $1.00 if it comes up heads, and loose $1.00 if it comes up tails. Player B
again does nothing. Which of the two coins does Player A flip? The answer depends on
the amount of capital Player A presently holds. If his capital X(t) is a multiple of 3 he
uses coin #3, otherwise he uses coin #2. Another way of saying X(t) is a multiple of
three is to write:

X (t ) mod 3 = 0

Bruce Graham 6 4/26/07

Now the key feature of the whole paradox is the magnitude and direction of the
unfairness of the two coins. For coin #2:

P (head ) = 3 − ε , P (tail ) = 1 + ε , ε << 1, typically ε = 0.005

4 4

This is good for Player A, he likes coin #2, but for coin #3:

P (head ) = 1 − ε , P (tail ) = 9 + ε , ε << 1, typically ε = 0.005

10 10

This is bad for Player A, he does not care for coin #3.

So clearly if Player A has a choice in playing Game B, the pure strategy of choosing coin
#2 is obvious. The choice does not depend on Player A’s wishes; it is dependent on the
present state of his capital modulo 3. Game B does not care how Player A got to this
level of capital or even the magnitude of his capital, but only if it is a multiple of three. A
naïve analysis of Game B might conclude that coin #3 would be used one third of the
time and coin #2 would be use two thirds of the time. This would be wrong! If we use
coin #3 it is likely we will loose and X(t) mod 3 goes from zero to 2 because we just lost
$1.00. On the next round we use coin #2 and it is likely we will win and X(t) mod 3 goes
from 2 back to zero. If we ever find ourselves in the state where X(t) mod 3 is equal to 1,
we use coin #2, it is likely that we win and we go to X(t) mod 3=2 where we spend the
bulk of our time bouncing back and forth between states zero and two.

Game B needs to be analyzed as a Discrete Markov Chain. We first define a state

variable Y(t) = X(t) mod 3 which takes values in the set {0,1,2}. For each state there is a
probability of being in that state. These three probabilities can be represented with a
vector as follows:

 P(Y (t ) = 0 )

 
V =  P(Y (t ) = 1) 
 P(Y (t ) = 2 )
 

Given the probability of being in a given state, we can use the rules for Game B to
compute the probabilities of being in a particular state after each round of Game B.
Playing Game B is equivalent to multiplying our probability vector by some constant
transformation matrix each round. How can we construct this matrix? For each state we
can be in, we know which coin must be flipped. If we know which coin is flipped, we
know the probability of heads and tails, and finally we know the state transition rules:

heads : state = (state + 1) mod 3

tails : state = (state − 1) mod 3

Bruce Graham 7 4/26/07

Note in particular that we cannot stay in the present state. Each column in our transition
matrix represents the probability of going from a present state to either of the two
possible successor states. We denote this matrix with a capital pi.

 0 1
+ε 3
−ε
 4 4

Π =  101 − ε 0 1
4
+ ε , where ε = 0.005
 9 +ε 3
−ε 0 
 10 4

The properties of the probability vector are determined by the algebraic properties of the
pi matrix. As epsilon is varied the eigenvalues and eigenvectors will change and imply
different behaviors. In this example the principal eigenvalue of this matrix is 1, which
means the system has a stationary solution. At some point

Π ⋅V = V

and further multiplications by pi have no effect on V. The other eigenvalues are a

complex conjugate pair with a magnitude less than one, which means things, cannot
diverge. I think this may be related to the transient behavior, but I was unable to confirm
this. Now, given an initial condition for the probability vector we can iterate as many
rounds of Game B as we like and see what happens. When we do this we notice three
things, there is an initial transient in the probability vector, after the transients disappear
all three probabilities approach stationary values, there is no dependence on initial
conditions.

Game B – Theoretical Analysis

To analyze Game B we need to compute the probability of using coin #3. Then we need
to compute the probability of winning using coin #3, plus the probability of winning if we
use coin #2. The probability of using coin #3, P0, is the same as the probability that
Y(t)=0.

P0 = 0.3846 − 0.2003 ⋅ ε = 0.3846 − 0.2003 ⋅ 0.005 = 0.3836

To compute the chance of winning at Game B we use the following expression:

Pwin = (1 − P0 ) ⋅ ( 34 − ε ) + (P0 ) ⋅ (101 − ε ) = 0.5 − 0.87 ⋅ ε = 0.5 − 0.87 ⋅ 0.005 = 0.4956

Game B is a losing game for all ε > 0.

The following MATLAB program multiplies an initial vector y by the transition matrix M as
many times as desired and keeps track of evolution of the probability vector in ym by
concatenating the new vector onto the previous vectors. In this example the array ym
will have three rows and a number of columns equal to nsteps+1. The array ym can be
easily plotted in MATLAB

Bruce Graham 8 4/26/07

MATLAB program dmc.m

% Explore Discrete Markov Chains

%
function ym = dmc(y,nsteps)
global epsilon
if(isempty(epsilon))
epsilon = 0.005 ;
end
global M
if(isempty(M))
M = [[0,.25+epsilon,.75-epsilon];
[0.1-epsilon,0,0.25+epsilon];
[0.9+epsilon,0.75-epsilon,0]] ;
end
ym = y ;
for n = 1:nsteps
y = M*y;
ym = [ym y] ;
end
end

Running this program confirms that P(Y(t)=0)=0.3836 in agreement with the theoretical
result. It also illustrates that Discrete Markov Chains go to a stationary condition from
any initial condition, so they cannot be chaotic! The initial condition was [1/3,1/3,1/3].

Bruce Graham 9 4/26/07

The Paradox
If we alternate the playing of Game A with the playing Game B in much the same fashion
that the Brownian ratchet is flashed ON and OFF the result is a net gain in capital. This
is written as an ordered pair, so [2,2] would be two rounds of Game A followed by two
rounds of Game B. Even more surprising than the net gain in capital from two losing
games is that we can alternate the games in a wide variety of ways, even choose which
game to play at random and the result is the same. The following simulation graph from
Parrondo’s website demonstrates this:

[3,2] is 3 games A followed by 2 games B

[2,2] is 2 games A followed by 2 games B
[4,4] is 4 games A followed by 4 games B
random is playing A and B in random order

The effect of the pi matrix can be seen in the transient at the beginning of playing Game
B alone. The transient is pretty much gone after 20 rounds. The paradox occurs when
for a given value of ε and an alternation strategy the probability of winning Game B is
less than ½, while the probability of winning the alternation of Game A and Game B is
greater than ½. Parrondo’s theoretical results are summarized as follows:

PwinB = 0.4956
PwinAB = 0.5079, ε = 0.005

Bruce Graham 10 4/26/07

Financial Application

Parrondo’s paradoxical result led some investigators to wonder if artificial traders could
succeed in producing a net gain in capital by alternately playing two losing games in the
marketplace. Bowman, Johansson and Lybäck investigated this possibility, but had little
success in identifying the games to be played.

They chose for their data 10 stocks over 252 trading days, starting March 1, 2000, in a
declining Swedish stock market. Each strategy started with the same capital and one
trade per day was allowed in which the strategy was allowed decide what to sell and
what to reinvest in. A summary of the strategies from their paper is shown in the table
below.

Strategy Description
Buy-and-Hold The buy and hold strategy acts as a control
(BaH) strategy that trades no stocks
Random This strategy trades stocks randomly
Insider The insider gets quality ex ante information
about some stocks on which it may react
before the market.
Buy low, sell high This Markovian value investor strategy
(BLSH) monitors if the stock has increased or
decreased in value during the latest time
interval. If the value has increased, it sells
the stock, and if the value dropped, it buys
the stock.
Buy low, sell random Like BLSH, except BRSH randomly chooses
(BLSR) what stock to sell
Buy random, sell high Like BLSH, except BRSH randomly chooses
(BRSH) what stock to buy
Buy High, sell low This Markovian trend investor strategy is the
(BHSL) opposite of BLSH.

Also present in their experiment was the passing of information messages, which the
artificial traders could act upon.

Bruce Graham 11 4/26/07

Each strategy had an initial capital of 10,000.00 units. The final results are reproduced
in the table below.

Strategy Value
BLSR 6110.88
Random 5524.60
BaH 5383.40
BLSH 5338.15
BHSL 5202.71
BRSH 5140.29

From their conclusions:

“For purposes of prediction, our results are almost useless, since we cannot in
general design in advance a portfolio of stocks, the prices of which are all
receding. In rare circumstances, such as during the period of almost universally
receding prices of IT stocks in the autumn of 2000, ex ante portfolios could
relatively easily be assembled, and then Parrondo variations would indeed be an
interesting alternative to buy-and-hold.”

and their objective:

“We intend to pursue the important question of strategy programming for artificial
traders, as we feel that such programming will be of increased importance in the
future. By replacing our unrealistic assumptions one by one, we hope to achieve
our ultimate goal of reasonably efficient strategies on real-time markets with non-
linear dynamics.”

Conclusions

If Bowman, Johansson, & Lybäck succeed in their objective, and implement their system
on a small scale, and nobody notices, they just might enjoy a comfortable retirement. I
doubt very much if they would publish that result. As soon as they succeed and their
success is noticed then other players will try to duplicate it, or regulators will forbid the
use of artificial traders, or some other factor will undoubtedly upset the delicate balance
required to make the Parrondo strategies work. I also believe that market makers and
floor traders who knew that there were artificial traders in play would devise means to
take advantage of their weak points, to the disadvantage of the artificial traders and their
sponsors.

Our coursework covered many types of complex behavior, each with distinct properties.
In this project the focus is on stochastic behavior which is time varying and random. As
complex as stochastic processes are they are definitely not chaotic. Instead of having a
sensitive dependence on initial conditions – in many cases they have no dependence on
them.

Bruce Graham 12 4/26/07

References

Feynman, R.P., Leighton, R.B., and Sands, M.,The Feynman Lectures on Physics,
Volume I, Addison-Wesley, 1963, pp.46-1 to 46-9.

von Neuman, J., Morgenstern, O.,Theory of Games and Economic Behavior,1944

Williams, J.D., The Compleat Strategyst, McGraw Hill, 1954

Parrondo,J.M.R., and Español,P., Criticism of Feynman’s Analysis of the Ratchet as an

Engine, American Journal of Physics. 64, 1125(1996)
Parrondo,J.M.R., Cisneros, B.J., Juegos paradójicos y máquinas térmicas brownianas.
Paradoxical games and Brownian thermal engines. (Submitted in Spanish), February
2000

Harmer, G.P., Abbott, D., Brownian ratchets and Parrondo’s games, Chaos, 11, #3,
September 2001, pp.705-714.

Boman, M., Johansson, S.J., Lybäck, D., Parrondo Strategies for Artificial Traders,
submitted to World Scientific April 26, 2002

URL’s

http://seneca.fis.ucm.es/parr/
http://monet.physik.unibas.ch/~elmer/bm/index.html
http://www-personal.engin.umich.edu/~bftsplyk/CS541/Parrondo

Bruce Graham 13 4/26/07