A Unified Approach to the Definition of Random Sequences

by
C. P. SCHNORR
Institut fiir Angewandte Mathematik
Universit/it Saarbriicken, West Germany
ABSTRACT
Using the concept of test functions, we develop a general framework within which
many recent approaches to the definition of random sequences can be described.
Using this concept we give some definitions of random sequences that are narrower
than those proposed in the literature. We formulate an objection to some of these
concepts of randomness. Using the notion of effective test function, we formulate a
thesis on the "true" concept of randomness.
1. The Concept of Test Function. Let X [X*] be the set of all infinite [finite]
bi nary sequences. A e X* denotes the empt y sequence. Ixl denotes the length
of x s X*. The concat enat i on of sequences x and y is described as the pr oduct
xy. This in an obvi ous way defines a pr oduct ABc X* u X ~ of sets A c X*
and B c X* u X ~. For a sequence x ~ X* u X ~ we denot e by x(n) the initial
segment of length n (x(n) = x if Ixl < n). The map ~: 2 x* -+ 2 x~ is defined by
~(A) = AX ~ (A c X*). Thr oughout the paper ~ will be the pr oduct measure
on X relative t o the probabi l i t y for I and 0.
Let us first explain the intuitive idea of r andom sequences, whi ch will be
discussed here. An infinite sequence is considered a r andom sequence if it
withstands all constructive stochasticity tests. Our mai n assumpt i on is t hat any
stochasticity test can be expressed by a funct i on F: X*- + R, where F( x)
indicates the extent t o whi ch the sequence x is susceptible t o the stochasticity
test F (R is the set of all reals). It seems nat ural t o t hi nk t hat F( x) is high
when x is susceptible t o the test and low otherwise. However, this is not in-
evitable. Let us give some examples.
Consi der a funct i on V: X* --~ R + t hat indicates the capital of a gambl er
when playing on bi nary sequences (R is the set of all non-negat i ve reals).
V( x) denotes the capital after the Ixlst trial when the sequence of the gambl i ng
system has the initial segment x. In a fair gambl i ng system the player' s gain has
t o satisfy the relation V(x) = 2- l ( V( xl ) + l~(x0)). It is nat ural to t hi nk t hat
lira sup, V(z(n)) < ~ if z is a r andom sequence. Consequent l y high values
V(z(n)) mean t hat the sequences z(n) are susceptible t o test V. Gambl i ng systems
of this kind were i nt roduced by J. Ville [15]. Ville proved t hat for every funct i on
V: X*- + R + satisfying V( x) = 2 -1 ( V( xO) + V(xl)) the set {z e X~[lim sup.
V(z(n)) = m } is a null set.
246
A Unified Approach to the Definition of Random Sequences 247
To give another example, we define a set U c Nx X* to be a sequential test
if Ui =aef {x ~ X* [(i, x) ~ U} satisfies
(1) U, = U,X*;
(:2) U~+~ = U~;
(3) ~(w,) <__ 2-' .
Then for every null set 9l c X ~ there is a sequential test U such that 9l c ~i~N
~o(U,). Relative to the sequential test U the critical level function mu: X*--->N
is defined by my(x) = max {mix ~ Urn). Consequently z withstands the sequen-
tial test U if and only if
lim sup mv(z(n)) < oo.
n
Hence m v reflects our intuition of a test function. A sequential test U is called
recursive if U c Nx X* is recursively enumerable (r.e.). These tests were
introduced by Mart i n-L6f [6]. A sequence z e X is random in the sense of
Mart i n-L6f if z Ni~N p(Ui) for every recursive sequential test.
In order to generalize the above mentioned examples, we define a function
F: X*--> R to be a constructive test if F satisfies the following properties,
which will be stated in an informal way.
(T1) F has to be constructive, i.e., F is to be given by algorithms.
(T2) There is a rule which assigns to F a null set gtF C X , the set of
infinite sequences which do not withstand the test F. Whether z s ~e
has to depend only on the sequence (F(z(n))ln e N).
The different definitions of random sequences we shall discuss here are
merely distinguished by specifying the above mentioned axioms more precisely.
We shall essentially consider two different rules according to (T2), namely:
(a) 9lF = {Z ~ X ~ llim sup F(z(n)) = oe },
n
(b) 91F = {Z ~ X~liim infF(z(n)) = oo}.
n
Because of (T1) the set of test functions of any fixed concept of test functions
can be enumerated. This implies that relative to any concept of test functions
satisfying the above mentioned axioms the following theorem is true.
THEOREM 1.1. The set of random sequences has measure 1.
Given a fixed concept of test functions, a test F is called universal if 91e D
gte, for any other test F~.
2. Martin-Liif Random Sequences Described by Martingales. Let Q be the
set of all rational numbers.
Definition 2.1. A (total) function F: X* --> R is called weakly computable
if there is a recursive function g: Nx X* ~ Q such that
g(i, x) <_ g(i+ 1, x) (i e N, x e X*)
lim g(i, x) = r( x) (x ~ X*).
i
248 C. P. SCHNORR
A funct i on F: X* -+ R is comput abl e in the usual sense i f F and - F are weakly
comput abl e.
A funct i on F: X* -+ R is said t o have t he martingale pr oper t y relative t o
t he probabilities f or 0 and I i f it satisfies t he condi t i on
(2.2) F( x ) = F(xO) + F(xl ) (x e X*).
These funct i ons are called martingales. In the actual case this means t hat the
ext ent t o which x withstands t he test F is the weighted average of t he ext ent
t o which x0 and xl wi t hst and t he same test.
The following l emma was pr oved by J. Ville [15].
LEMMA 2.3. I f F: X* -+ R + satisfies (2.2), then the set 91 = {z e X~ [
lim sup. F( z( n) ) = oo} is a nul l set.
Pr o o f We define f or k s N:
F k = {x e X ' I F ( x ) > k }
F k = {x e r k Ix Fk XX* }.
This implies F k n l r k XX * = o. Ir k consists of all t hose sequences in F k which
have no initial segment in Fk. We have /z~0(F~)=/z~o(Irk) = ~x~r~ 2-1xl. It
follows f r om (2.2) t hat
F(A) > ~ r ( x ) 2 -Ixl > k Z 2-1xl > kt zg(rk)
x~lek x~Fk
Consequent l y,
( 2 . 4 ) ~(r k) <- r(A)k -1
Since 9l c (~k~Nq~(Fk), this proves t hat 9l is a null set.
We are now able t o present our first example of a concept of a test funct i on.
Definition 2.5. A t ot al funct i on F: X* -+ R + is a (1)-test i f it is weakly
comput abl e and satisfies (2.2). The set of infinite sequences whi ch do not with-
st and t he (1)-test F is defined t o be
91r = {z e X~llim sup F( z ( n) ) = oo}.
Definition (2.5) is justified by the following t heorem.
THEOREM 2.6. An infinite sequence wi t hst ands all (1)-t est s i f and onl y i f
it is random in t he sense o f Mart i n- L6f i
First we will pr ove a lemma.
LEMMA 2.7. For every r.e. set A c X* one can ef f ect i vel y const ruct a recursive
set B c X* such t hat A X * = B X * and B n B X X * = ~, i.e. B is pref i x-f ree.
Proof . Let A be given by a recursive funct i on h: N--> X* u {I} such t hat
A = h ( N) n X* . (The symbol I has t o be used i f A is empt y. ) We denot e A, =
Ui <, h( i ) (~ x * . r(n) is t o be t he maxi mal length of sequences in A,, r(n) = 0
if A, = ~. The i ndi cat or funct i on I , : X* --> {0, 1 } of the set B is defined as
A Unified Approach to the Definition of Random Sequences 249
follows: IB(x) = 0 for all xE X* such t hat Ix I ~ r( n) +n for all n ~ N. IB(x)
with I x l = r(n)+n is defined recursively. For [ x l = r ( 0 ) :
/~(x) = {10 x = h(O)
otherwise,
and for Ixl = r(n)+n, n > 1:
{10 x s h ( n ) X * - A " - l X *
l~(x) otherwise.
The const ruct i on implies t hat B is prefix-free. In addi t i on we have An X* =
(B n {x I Ixj < r( n) +n}) X*. Hence AX* = BX*, as desired.
In t he following pr oof of (2.6) we shall assume t hat a recursive sequential
test U is given by an r.e. set V c Nx X * such t hat V i n Vi XX* = o and
V, X* = U, (i ~ N) .
Proof of(2.6). (1) Let a recursive sequential test U be given an r.e. set V c
N X* as above. We define the (1)-test F: X* ~ R + as follows:
r ( x ) = 2 i [ Z 2-Iwl + Z 1
i~lq ~xy~V~ x(n)~vt I
n<lxl /
It is easy t o verify t hat F satisfies the rel at i on (2.2). One has onl y t o consider
the cont ri but i ons t o F(x), F(xO), F( xl ) which result f r om y ~ Vi. Since Vi is
prefix-free, we have F ( A) = ~Ntzq~(Vi). Therefore F(A) is bounded. Hence
F is a funct i on F: X* -+ R . Fr om the definition of F it follows immediately
t hat F is weakly comput abl e. Let us suppose now t hat z ~ q~(Vi). Hence there is
an n such t hat z(n) ~ V i. This implies F(z(n)) > i. It follows t hat ("]i~N~o(V~)
C ~ ' ~ F "
(2) Let z be r andom in the sense of Mar t i n- L6f and let F: X* ~ R be a
(1)-test. Choose k > F(A) and define V = Nx X* as follows:
V, = {x ~ X' I F( x ) > 2' k}.
Since F is weakly comput abl e, V is r.e. Because of (2.4) we have/~o(Vi) _< 2- i .
Hence a recursive sequential test U can be defined by U~ = V~X*. Fr om
z N~u~(U~) it follows t hat z withstands the test F.
Perhaps it is interesting t o not e t hat t he existence of a universal (1)-test
follows f r om a simple argument . Let (F~ li ~ N) be a recursive enumerat i on of all
(1)-tests with Fi(A) _< 1. Hence F = ~i ~u2-i Fi is a universal (1)-test.
It should be not ed t hat we can use lim i nf as well as lira sup in the definition
of 92v f or a ( 0- t est F.
LEMMA 2.8. Let F be a universal (1)-test. Then the following equivalence holds
f or all z ~ X~:
lira i nf F(z(n)) = ~ <=~lira sup F(z(n)) = co.
t l 1
Proof Let U c N x X* be a universal recursive sequential test t hat is given
by a r.e. set Vc Nx X* as in par t (1) of the pr oof of (2.6). Consi der the (1)-test
250 C.P. ScrINOgg
F as defined in par t (1) of the pr oof of (2.6). It will suffice t o show t hat
z ~ Ai~N~o(Vi) implies lim, F( z ( n ) ) = oo.
Suppose z ~ A i~N~(Vi). Then t o i ~ N t here exists n E N such t hat z ( n) ~ Vi.
This implies F ( z ( m) ) >_ i f or all m _> n. Since this holds for every i ~ N, (2.8) is
proved.
3. An Objection to Randomness in the Sense of Martin-Liif. The algorithmic
st ruct ure of a (1)-test F is not symmetrical. There is no reason why a mart i ngal e
F shoul d be weakly comput abl e and - F shoul d not be so. Taki ng this i nt o
consi derat i on we make the following definition.
Definition 3.1. A funct i on F: X* ~ R is a (2)-test i f it satisfies (2.2) and
i f - F is weakly comput abl e. The set of sequences t hat do not withstand the
test F is defined t o be
9~F = {z e X 1 lim sup F( z ( n ) ) = oo }.
i1
We consider t he quest i on whet her (1)-randomness is equivalent t o (2)-
randomness. Ther e seems t o be an obj ect i on t o either of these concepts of
randomness because this is not true.
THEOREM 3.2. Ther e e x i s t sequences whi ch are ( 2) - r andom a n d whi ch are
not ( 1) - r andom.
We will first pr ove a lemma.
LEMMA 3.3. L e t F be a ( 2) - t es t a n d a > 0 a r at i onal number . Then t here
e x i s t s a recursi ve z ~ X ~ such t hat F( z ( n ) ) < F ( A) + a (n E N) . Thi s me ans t hat
z ~ .
Pr oof . Let the (2)-test F be given by a recursive g: Nx X* ~ Q such t hat
g( i , x ) >_ g ( i + 1, x) and lim sup, g( i , x ) = F( x ) . Let b be rat i onal with F(A)
- a / 2 < b < F(A). The sequence z will be const ruct ed recursively as follows.
Assume t hat z ( n) has been const ruct ed such t hat
F ( z( n) ) <_ b + a 2- i - 1 (i <_ n).
j =0
( Not e t hat this assumpt i on is trivial f or n = 0.) Consequently, t here exists an
x e X with
F( z ( n ) x ) <_ b + a ~ 2 - j - 1.
j =0
Hence we can effectively det ermi ne i and x such t hat
n+l
g ( i , z ( n ) x ) <_ b + a ~, 2 - J - 1 .
j =0
Define z ( n + 1) = z ( n) x . Fr om the const ruct i on it follows t hat
F( z ( n ) ) < b + a <_ F ( A) + a (n ~ N).
P r o o f o f ( 3 . 2 ) . Let (F~ [i ~ N) be an enumerat i on of all (2)-tests with F~(A) < I.
A Unified Approach to the Definition of Random Sequences 251
It will suffice t o define a sequence z which is not (1)-random and satisfies
lira sup, Fi ( z( n) ) < ~ f or all i ~ N.
Let F be a universal (1)-test, i.e. X - ~F consists precisely of all (1)-random
sequences, z s X ~ will be defined inductively. Assume t hat Z( nk) with nk ~ N
is al ready defined such t hat F( z ( nk) ) > k , and t hat
k k
2 - " ' - ' F , ( z ( j ) ) < ~ 2 - ' + 1 ( j < nk).
i =0 i =0
(Not e t hat t he i nduct i on hypothesis is trivial f or k = 0, no = 0 and F o ( A ) < 1.)
To per f or m t he i nduct i on step, we consider Fk+l . We obvi ousl y have
2 - " k - k Fk +l ( z ( n k ) ) < 2 -k. This follows f r om Fk +I ( A) < 1 and t he martingale
propert y. Consequent l y t here is a recursive y~ X such t hat y(nk) = z ( n ~
satisfying f or every nk+ 1 >- nk t he rel at i on
k+l k+l
2 - " ' - ' F i ( y ( j ) ) <_ ~ 2 -`+1 ( j ~ N ) .
f =0 f=O
This essentially follows f r om the const ruct i on used in t he pr oof of (3.3).
Since y is recursive t here is a nk+ 1 > n k such t hat F ( y ( n k +l ) ) > k + l .
Define z(nk+ 1) = Y(nk+ 1).
The definition of z implies t hat z ~ 9~F. On the ot her hand we have
lim sup ~ 2- "' - f F i ( z ( j ) ) < 4 (i ~ N ) .
j / =0
Consequent l y z ~e~ (i ~ N).
We r emar k t hat it is not difficult t o prove t hat every (1)-random sequence
is also (2)-random. (1)-randomness is a nar r ower concept of r andom sequences
t han (2)-randomness. I t seems surprising t hat f or a mart i ngal e F it is i mpor t ant
whet her we choose F or - F t o be weakly comput abl e.
Now we aim at developing a concept of randomness based on martingales
whose algorithmic st ruct ure is symmetrical.
Definition 3.4. A mart i ngal e F: X* ~ R + is a (3)-test i f t here is a recursive
funct i on g: Nx X* ---> Q such t hat limi g( i , x ) = F ( x ) ( x ~ X * ) . The set of
sequences t hat do not wi t hst and a (3)-test F is defined t o be
~R~, = {z ~ X~llim F( z ( n ) ) = 00}.
We will prove t hat (3)-randomness is considerably nar r ower t han (1)-
randomness. Obviously every (1)-test is also a (3)-test. Therefore every (3)-
r andom sequence is a (1)-random sequence. To prove t hat t he converse does not
hold, we consider t he Kl eene hi erarchy of sets.
The Kleene hi erarchy of predicates classifies t he "ar i t hmet i cal " sets in
classes Z. , 17. (n = 0, 1,. .) defined as follows. ~. is the class of all sets A of
t he f or m A = { a l ( Qa x a ) ( Q 2 x 2 ) . . . ( Q . x . ) P ( a , x . x z , . ' . , x . ) } , where P is a
recursive predicate, t he Q2k+l are existential quantifiers and t he Q2k are
universal quantifiers. I I . is t he class of all sets as above, except t hat t he Q2k+ 1
252 C. P. Scn~oP, a
ar e uni ver sal quant i fi ers and t he Q2k ar e exi st ent i al quant i fi ers. The f ol l owi ng
fact s ar e k n o wn (see Davi s [17]).
(1) X o = rl o = z 1 c~ 1-I 1 is t he col l ect i on o f all recursi ve sets.
(2) A e E. ~ A c ~ l-I,, (A c is t he c ompl e me nt o f A. )
(3) Z, u I I , c Z. + l ~ I I , +~ f or all n > 0 a nd c ont a i nme nt is pr ope r f or n > 0.
(4) A E Y~.+ 1 ~" A is r ecur si vel y enumer abl e rel at i ve t o a set B s I I . .
(5) A e Z. + 1 c~ I I , +1 ~ A is a recursi ve rel at i ve t o a set B s I I , .
The class I-I. n Z, is usual l y de not e d by A. A sequence zE X ~ is t o be i n Z.
[1-I,] i f { n [ z n = 1} is i n Z, [I-I,].
L E MMA 3.5. T h e r e e x i s t s e que nc e s in A 2 whi ch a r e ( 1) - r andom.
P r o o f Let F: X* - + R + be a uni ver sal (1)-test t ha t is gi ven by a recursi ve
f unct i on g: Nx X* - + Q. We s uppos e t hat F( A) < 1. The n t he f ol l owi ng
pr edi cat e P: X* ~ { 0 , 1} is i n Ha: P ( x ) = 1 ~ Vi~Ng(i, x) < 1. Gi ven P one
can cons t r uct z s X ~ recursi vel y as f ol l ows : zi + 1 = 1 ~ : . P ( z ( i ) l ) = 1. Hence
it f ol l ows f r o m t he a bove pr ope r t y (5) t ha t z is i n A z. The c ons t r uc t i on i mpl i es
t ha t z 9~F.
To compl et e t he p r o o f o f our asser t i on t ha t ( 3) - r andomnes s is cons i der abl y
na r r owe r t ha n ( 1) - r andomness, we est abl i sh t he f ol l owi ng t heor em.
T HE OR E M 3.6. The r e do n o t e x i s t s equences in Z 2 u I I 2 whi c h a r e (3)-
r a n d o m.
P r o o f (I) Let z be a sequence i n Z 2. Ac c or di ng t o t he defi ni t i on this means
t ha t { n l z . = 1 } is i n ~2. The n t her e exists a recursi ve pr edi cat e P: N 3 - + {0, 1 }
such t hat , f or all n e N, z, = 1 ~ 3j~NVi~NP(j, i, n) = 1. We define a (3)-test F
sat i sfyi ng z e 92v by speci f yi ng a recursi ve f unct i on g: Nx X* ~ Q. We de not e
f ( i , n) = {j[ VP ( L r, n) = 1 ; j < i}.
r <i
The finite set f ( i , x ) can be effect i vel y det er mi ned. The n we c omput e g( i , x )
r ecur si vel y as f ol l ows : g( i , A) = 1 ( i e N ) . I f ( f ( i , n) ~ A y, = 1) v ( f ( i , n)
= ~ A y, = 0) we define g( i , y( n) ) = 2g( i , y ( n - 1)) ( y e X). I f ( f ( i , n) # ~ A
y , = O ) V ( f ( i , n) = e Ay . = 1) we define g( i , y( n) ) = 0 ( y e X ~ ) . Thi s
i mpl i es t ha t
2 l i m g( i , y ( n - 1)) z, = y, .
lira g( i , y( n) ) = i 0 z , ~ y , .
Hence F is a (3)-test a nd it f ol l ows t ha t
F ( y ( n ) ) = { 2 0 " y ( n ) = z ( n )
y ( n ) # z ( n) .
( I I ) Let z be an el ement o f 112. Thi s means t ha t { n l z ~ = 0) is i n Z v Hence
t he above a r gume nt al so i mpl i es t ha t z is not ( 3) - r andom.
4. Random Sequences and the Concept of Minimal Program Complexity. We
shal l pr ove t ha t t he concept o f r a ndomne s s t hat has been pr opos e d by Love l a nd
[5] is na r r owe r t ha n t he defi ni t i on o f r a n d o m sequences by Mar t i n- L6f .
A Unified Approach to the Definition of Random Sequences 253
Let A : X* N ~ X* be a part i al recursive funct i on satisfying A ( x , n) ~ X "
f or all (x, n) in t he domai n of A. Kol mogor of f [4] defined t he condi t i onal
compl exi t y Ka ( x l n ) of a sequence x with length n relative t o the al gor i t hm A:
' ~ i f A ( y , n) ~ x f or all ~ X*,
Y
Ka( x [ n) = ~mi n { [y] I A( y, n) = x} otherwise.
I f A ( y , n) = x : t hen y is called a pr ogr am to comput e the sequence x by t he
al gor i t hm A.
Definition 4.1. A funct i on F: X--~ Q is called a (4)-test i f t here is a p. r.
funct i on A: X * N - - + X * as above, such t hat F ( x ) = n - K a ( x ] n ) f or all
sequences x of length n. T t v = {z~ X~llim inf, F( z ( n ) ) > ~ } is the set of
sequences t hat do not wi t hst and the (4)-test F.
Thi s definition of r andomness has been explicitly pr oposed by Lovel and
in t er ms of the uni f or m compl exi t y [5]. (4)-randomness means t hat f or every
al gor i t hm A t here exist infinitely ma ny initial segment s wi t h high pr ogr am
compl exi t y. The ori gi nal i dea was t hat every r a ndom sequence z woul d satisfy
l i m SUpn ( n - - KA( z ( n ) l n ) ) < ~ f or every al gori t hm A. However it was shown by
Mar t i n- L6f [7] t hat t here exist no sequences t hat satisfy this pr oper t y.
Mar t i n- L6f [7] pr oved t hat ~Re is a null set f or every (4)-test F. Hence,
Defi ni t i on 4.1 satisfies our axi oms (T1) and ( T2 ) o f a test funct i on. Mor eover ,
it is known f r om [7] t hat every ( 4) - r andom sequence is r a ndom in the sense of
Mar t i n- L6f . We shall pr ove t hat the converse of this t heor em does not hold.
T HE ORE M 4.2. Ther e e x &t s a sequence z ~ X a n d a p. r . f u n c t i o n A :
X * x N --> X * such t hat l i m, ( n - Ka( z ( n) l n) ) = oo a n d z i s a Ma r t i n - L 6 f r andom
sequence.
P r o o f Let F: X* --> R + be a uni versal (1)-test wi t h F(A) < 1 t hat is given
by t he recursive funct i on g: N x X* - + Q. We consi der the (1)-random sequence
z E A 2 defined as in the pr oof of Le mma 3.5. P: X* ~ {0, 1 } is the fol l owi ng
predi cat e in 17 a : P ( x ) = 1 .~> Vi~Ng(i, x) < 1. Gi ven P we define z recursively
as fol l ows: z i +l = mi n { j ~ X l P ( z ( i ) j ) = 1}. z is ( 1) - r andom and we const ruct
a p. r. funct i on A: X * x N - - - > X * such t hat l i m, ( n - K a ( z ( n ) ] n ) ) = oo.
Let h: N ---> N x X* x Q be the recursive funct i on defined by h ( N) = {(i, x,
q)[g(i , x) = q}. Ther e exists a p. r. funct i on A: X* x N- + X* such t hat , f or all
x ~ X * , A ( x , i + Ix[) = r(i, x ) x . Her eby we define r( i , x ) ~ X i recursively as
fol l ows:
r(O, x) = A (x ~ x* ),
with
r ( i + 1, x ) = r( i , x ) s ( i , x )
a ndr n _< Ix[ =~q _< 1 "
Her eby we suppose t hat mi n ~ = 1.
Usi ng this const ruct i on it can easily be pr oved t hat f or every i ~ N t here
exists an n, ~ N such t hat , f or all x ~ X* wi t h [x[ >_ n,, A( x , i + [x]) = z ( i ) x .
Consequent l y l i m, ( n - Ka( z ( n ) ]n)) = oo.
254 C. P. SCHNORR
It is obvi ous t hat t here is an inconceivable multiplicity of possibilities t o
define test funct i ons satisfying conditions (T1), (T2). Therefore one probl em
still remains unsolved, which we shall discuss later. It deals with the quest i on
which concept among all possible concepts of test functions is t he really " t r ue "
one and i f such a distinguished concept exists at all.
5. An Objection to the Concept of (4)-Randomness. We shall formul at e an
obj ect i on t o the concept of (4)-randomness, al t hough (4)-random sequences
possess all st andard statistical propert i es of randomness such as the law of
large numbers and t he law of the iterated logarithm. Our mai n obj ect i on shall
be discussed later. It concerns our feeling t hat properties of randomness are
i mposed on (4)-random sequences t hat have no physical meaning.
Anot her difficulty arises f r om the fact t hat t here is no analogue to t he
mart i ngal e pr oper t y (2.2) regarding (4)-tests.
This lack of an anal ogue t o (2.2) has the consequence t hat a (4)-random
sequence z has infinitely many initial sequences z(n) with high values F(z(n)) rel-
ative t o a universal (4)-test F (high values F(z(n)) mean low complexity of z(n)).
This follows f r om an argument by Mar t i n- L6f [7] which, when applied to (4)-
randomness, shows t hat t here exists no infinite sequence z such t hat lira supn
F(z(n)) < oo. Because of Lemma 2.8 this effect is excluded as regards (1)-
randomness.
It seems nat ural t o define a hi erarchy of complexity for infinite sequences
as follows (in a similar manner this has been done by Lovel and who used t he
uni f or m compl exi t y measure [5]). Let F be a universal (4)-test. Its existence is
proved in [4] and [14]. For every non-decreasing funct i on f we denot e
(5.1) Cs =d a {Z ~ X ~ [lim i nf ( F( z ( n) ) - f ( n) ) < oo}.
n
I f f is bounded, t hen Cy is exactly the set of all (4)-random sequences. For a
slowly increasing unbounded funct i on f the sequences in Cy are expected t o be
approxi mat el y r andom. This, however, is by no means true. On the cont rary,
we can prove the following t heor em which also holds relative t o (4)-random
sequences and for the concept s of randomness derived f r om the uncondi t i onal
(cf. [4]) and also for uni form compl exi t y measures.
THEOREM 5.2. Let f : N--+ N be a non-decreasing unbounded function.
Then there exists a sequence in Cf that does not satisfy the law of large numbers.
Fr om the statistical poi nt of view t he law of large numbers is one of the
most basic laws of randomness. There are very simple statistical tests which
reject sequences not satisfying this law. Ther ef or e the sequences in Cf, even
with a very slowly increasing unbounded f , cannot be viewed as approxi mat el y
random. And this is an obj ect i on t o (4)-randomness.
We r emar k t hat Theor em 5.2 solves the question of Lovel and [5] whether
t here exists a non-decreasing unbounded f such t hat C; is precisely the set of
all r andom sequences. Because of Theor em 5.2 such a funct i on cannot exist.
Proof. Let f : N- + N be an unbounded non-decreasing funct i on and let
F be a universal (4)-test. We define z = z l z 2 . . . z i . . . G X ~ by induction.
Suppose t hat z(ni) is already defined. Then we set Zk = 1 f or n i < k <_ 2n i.
It is obvi ous t hat t here exists a (4)-random sequence x such t hat the initial
A Unified Approach to the Definition of Random Sequences 255
sequence x(2ni) is equal t o z(2ni). Then t here is an m > 2hi such t hat F(x(m)) <
f(m), since otherwise x would be not (4)-random. We define z(m)= x(m).
Now t ake m f or n~+ 1 and proceed by induction. The const ruct i on obvi ousl y
implies t hat z E C:, and on the ot her hand t he relation lim, ( l / n) ~7=xzi =
cannot be satisfied.
Remark. Theor em 5.2 also holds in this f or m f or t he concept of (1)-random-
ness, t hat is, if F in (5.1) is a universal (1)-test. However, this is no longer an
argument against this concept of r andom sequences. Since we have used lim sup
in the definition of (1)-tests, the appropri at e hi erarchy of sequences relative t o
the concept of (1)-randomness has t o be defined as follows, where F is a uni-
versal (1)-test and f : N ~ N is a non-decreasing funct i on:
(5.3) K: = {z ~ X~llim sup (F(z(n))-f(n)) < oo).
n
Fr om a t heor em in Schnorr [11] it follows t hat all sequences in K: satisfy t he
law of large numbers i f f(n) increases less t han any exponent i al funct i on
a"(a > 1). On t he ot her hand, Theor em 5.2 is not a suitable definition relative
t o a universal (4)-test F. It follows f r om an argument by Mar t i n- L6f t hat in
this case K: is empt y f or all slowly increasing functions f For instance, the
rel at i on ~,%1 2- : ( ") = m implies t hat K: = ~ [7].
6. On the Tr ue Concept of Randomness. The deficiency residing in the
previous concepts of randomness is, in our opinion, t hat properties of r andom
sequences are post ul at ed which are of no significance to statistics. Many
insufficient approaches have been made until a definition of r andom sequences
was pr oposed by Mar t i n- L6f which f or t he first time included all st andard
statistical properties of randomness. However, t he inverse post ul at e now seems
t o have been violated.
The acceptable definition of r andom sequences cannot be any f or mul at i on
of recursive funct i on t heory which contains all relevant statistical propert i es
of randomness, but it has t o be precisely a charact eri zat i on of all those propert i es
of randomness t hat have a physical meaning. These are intuitively those
propert i es t hat can be established by statistical experience. This means t hat a
sequence fails t o be r andom in this sense i f and onl y i f t here is an effective
process in which this failure becomes evident. On the ot her hand, it is clear
t hat i f t here is no effective process in which t he failure of the sequence t o be
r andom appears, t hen the sequence behaves like a r andom sequence. Therefore
the definition of r andom sequence has t o be made in such a way t hat this
sequence is r andom by definition.
In a series of papers [9, 10, 11, 12] we tried t o render clear this intuitive
not i on of randomness. It t urns out t hat t here are rat her different approaches
t o this concept of which all lead t o equivalent definitions. This paper has been
written t o give a comprehensi ve appr oach by test functions.
Fr om the poi nt of view of test funct i ons one woul d consider a sequence t o
be r andom i f it stands all effective tests. But how are the effective tests to be
defined ? It is nat ural to post ul at e t hat an effective test F: X* ~ R is comput abl e
in the ordi nary sense instead of being merely constructive.
256 C. P. SCHNORR
Definition 6.1. A funct i on F: X* ~ R is comput abl e i f t here is a recursive
funct i on g: Nx X* -+ Q such t hat [g(n, x ) - F ( x ) [ < 2 - " ( x ~ X * , n ~ N) .
Our considerations in Section 5 should have made clear t hat a reasonabl e
concept of test funct i on has t o include t he mart i ngal e pr oper t y (2.2). Com-
put abi l i t y and t he mart i ngal e pr oper t y suffice t o characterize effective tests.
But which sequences are refused by an effective t est ? In anal ogy t o (2.3) one
would define t hat a sequence z does not wi t hst and t he test F i f and onl y i f
lim sup, F( z ( n ) ) = ~ . However, i f t he sequence F( z ( n ) ) increases so slowly t hat
no one worki ng with effective met hods onl y would observe its growt h, t hen
the sequence z behaves as i f it withstands t he test F. The definition of ~RF has
t o reflect this fact. That is, we have t o make const ruct i ve the not i on lim sup,
F( z ( n ) ) = oo.
Definition 6.2. Let f : N -+ N be a funct i on. We write k lim sup, f ( n ) = 0%
i f and onl y i f t here exists a recursive, monot one and unbounded funct i on
g: N ~ N such t hat lim sup, ( f ( n ) - g ( n ) ) > O.
Now we present our concept of effective tests [10].
Definition 6.3. A funct i on F: X* ~ R + is an effective test i f it is comput abl e
and satisfies the mart i ngal e propert y. The set of sequences t hat do not with-
st and the test F is defined t o be ~llr = {z e X ] k lim sup, F( z ( n ) ) = oo}.
A sequence is called (0)-random i f it withstands all effective tests. It is our
thesis t hat (0)-randomness characterizes all relevant statistical propert i es of
r andom sequences. To confi rm this thesis we shall restate some results f r om
earlier papers.
In [11] we established t wo interesting classifications of t he propert i es of
r andom sequences. A law of stochasticity is called of order ] ( f : N ~ N is a
non-decreasi ng funct i on) i f there is an effective test F: X* ~ R + such t hat
(6.4) lim sup ( r ( z ( n ) ) / f ( n ) ) > 0
f or all z e X ~ t hat do not satisfy this law.
The growt h of the funct i on f indicates t he i mport ance of t he law under
consideration. It is shown in [11] t hat the law of large numbers has a rapi dl y
growing or der funct i on. This is in har mony wi t h the fact t hat t he law of large
numbers is certainly one of t he most basic laws of probability. It is also shown
in [11] t hat the class of laws having t he same order as t he law of large numbers
is a very reasonabl e class. As is the case with some ot her concept s of randomness,
the set of recursive sequences can be charact eri zed in terms of test functions.
A sequence is recursive i f and onl y i f it does not satisfy some law of order
f ( n ) = 2" (as will be proved in [16]).
The above results relative t o t he or der of a law hol d f or effective tests, as
well as f or (1)-tests, i f a suitable f or mul at i on is chosen. However, the second
classification of tests which is based on the compl exi t y of test funct i ons (regard-
ing t he amount of time and space t o comput e t hem [3]) is meaningful f or
effective tests only.
Ther e is no universal effective test. To every effective test F a recursive
sequence not in Tt~ can be constructed. To every effective test F t here exists
A Unified Approach to the Definition of Random Sequences 257
an equivalent effective test F" ,* ~ Z(2) t hat is recursive (Z(2) is the set of
all finite dual fractions in R+). Let us consider a fixed complexity class (accord-
ing the amount of time or space) of effective test functions F: Y* ~ Z(2).
One can construct recursive sequences which will stand every effective test of
a fixed complexity class [11].
An infinite sequence z has a certain degree of (0)-randomness i f it stands all
tests of a corresponding complexity class. The degree of (0)-randomness yields
a classification of recursive sequences. Sequences t hat have a certain degree of
(0)-randomness have very interesting properties ([11], [12]).
It is the opinion of the aut hor t hat this classification of recursive sequences
relative t o their degree of (0)-randomness is an i mport ant argument for the
concept of effective tests. The existence of (1)-random sequences as well as t hat
of (0)-random sequences can be proved by non-constructive methods only.
These sequences exist onl y by virtue of the axiom of choice. However, we can
approximate the behaviour of (0)-random sequences by constructive methods.
This does not hold for (I)-random sequences. Because of Theorem 3.2, (1)-
randomness is not equivalent to (0)-randomness (see also [9]).
Anot her i mport ant argument for our thesis which proposes (0)-randomness
as the "really t rue" concept of randomness is t hat some different approaches
lead to an equivalent definition. It is proved in [10, Part I] t hat a sequence is
(0)-random i f and only i f it is not cont ai ned in any null set in the sense of
Brouwer. This concept of null set is current in constructive analysis (see, for
instance, [1]) and dates back t o an intuitionistic formul at i on of L. E. J. Brouwer
[2], A reasonable characterization of (0)-random sequences by their program
complexity can be found in [16]. But what is most surprising is t hat the original
ideas on which von Mises based his concept of collective can be modified t o
characterize exactly the (0)-random sequences. It is shown in [10] t hat a sequence
z is (0)-random i f and onl y i f every sequence y which is an image of z under a
constructive measure-preserving map H: X Q ~ X ~ satisfies the law of large
numbers. This means t hat the concept of Stellenauswahl in [8] has to be replaced
by the not i on of constructive measure-preserving map. All these and some
addi t i onal results will be included in [16].
REFERENCES
[1] E. BISHOP, Foundations of Constructive Analysis, McGraw-Hill, New York, 1967.
[2] L. E. J. BROUWER, Begriindung der Mengenlehre unabh/ingig vom logischen Satz vom
ausgeschlossenen Dritten. Zweiter Teil. Verh. Niederl. Akad. Wetensch. Afd. Nat uurk.
Sect. I 12 (1919), 7.
[3] J. HARTMANIS and R. E. STEARNS, On the comput at i onal complexity of algorithms,
Trans. Amer. Math. Soc. 117 (1965), 285-306.
[4] A. N. KOLMOGOROFF, Tri podhoda k opredeleniju ponjatija "koli~estvo informacii",
Problemy Pereda~i lnformacii 1 (1965), 3-1 I.
[5] D. LOVELAND, On mi ni mal program complexity measures, ACM Symposium on Theory
of Computing, pp. 61-66, May 5-7, 1969.
[6] P. MARTIN-LOF, The definition of r andom sequences, Information and Control 9 (1966),
602-619.
[7] P. MARTIN-L6F, Complexity oscillations in infinite bi nary sequences, unpublished.
258 C. P. SCHNOIU~
[8] R. VON MiseS, Grundlagen der Wahrscheinlichkeitstheorie, Math. Z. 5 (1919), 52-99.
[9] C. P. SCHNORR, Eine Bemerkung zum Begriff der zut"alligen Folge, Z. Wahrscheinlichkeits-
theorie Verw. Gebiete 14 (1969), 27-35.
[10] C. P. SCrtNORR, Ober die Definition von effektiven Zufallstests, I - I I , Z. Wahrscheinlich-
keitstheorie Verw. Gebiete 15 (1970), 297-312, 313-328.
[11] C. P. SCHNORR, Klassifikation der Zufallsgesetze nach Komplexit~it und Ordnung,
Z. Wahrscheinlichkeitstheorie Verw. Gebiete 16 (1970), 1-21.
[12] C. P. SCHNORR, 13ber die Zufiilligkeit und den Zufallsgrad von Folgen, Symposium on
Formale Sprachen und Automatentheorie, Oberwolfach, 1969.
[13] E. SPECKER, Nicht konstruktiv beweisbare S~itze der Analysis, 3". symb. Logic 14 (1949),
145-158.
[14] R. J. SOLOMONOFF, A formal theory of inductive inference, I, Information and Control
7 (1964), 1-22.
[15] J. VILLE, l~tude Critique de la Notion de Collectif, Gauthiers-Villars, Paris, 1939.
[16] C. P. SCHNORR, Zuf~illigkeit und Wahrscheinlichkeit, Lecture Notes in Mathematics (in
preparation).
[17] M. DAvis, Computability and Unsolvability, McGraw-Hill, New York, 1958.
(Received 24 April 1970, and
in revised f arm 11 August 1970)