1/30

Discrepancy and SDPs

Nikhil Bansal (TU Eindhoven)
Outline
Discrepancy: definitions and applications
Basic results: upper/lower bounds
Partial Coloring method (non-constructive)

SDPs: basic method
Algorithmic Spencers Result
Lovett-Meka result
Lower bounds via SDP duality (Matousek)
2/30
Material
Classic: Geometric Discrepancy by J. Matousek

Papers:
Bansal. Constructive algorithms for discrepancy minimization,
FOCS 2010
Matousek. The determinant lower bound is almost tight
Lovett, Meka. Discrepancy minimization by walking on the
edges

Survey with fewer technical details:
Bansal.
3/30
4/30
Discrepancy: What is it?
Study of gaps in approximating the continuous by the discrete.

Original motivation: Numerical Integration/ Sampling

Problem: How well can you approximate a region by discrete points
Discrepancy:
Max over intervals I
|(# points in I) (length of I)|
5/30
Discrepancy: What is it?
Study of gaps in approximating the continuous by the discrete.

Problem: How uniformly can you distribute points in a grid.

Uniform : For every axis-parallel rectangle R
| (# points in R) - (Area of R) | should be low.
n
1/2
n
1/2
Discrepancy:
Max over rectangles R
|(# points in R) (Area of R)|
6/30
Distributing points in a grid
Problem: How uniformly can you distribute points in a grid.

Uniform : For every axis-parallel rectangle R
| (# points in R) - (Area of R) | should be low.
Uniform
Random
Van der Corput Set
n= 64
points
n
1/2
discrepancy n
1/2
(loglog n)
1/2

O(log n) discrepancy!
Quasi-Monte Carlo Methods
With N random samples: Error \prop 1/\sqrt{n}
Quasi-Monte Carlo Methods: \prop Disc/n

Can discrepancy be O(1) for 2d grid?
No. \Omega(log n) [Schmidt ]

d-dimensions: O(log^{d-1} n) [Halton-Hammersely ]
\Omega(log^{(d-1)/2} n) [Roth ]
\Omega(log^{(d-1)/2 + \eta} n
[Bilyk,Lacey,Vagharshakyan08]



7/30
8/30
Discrepancy: Example 2
Input: n points placed arbitrarily in a grid.
Color them red/blue such that each rectangle is colored as
evenly as possible

Discrepancy: max over rect. R ( | # red in R - # blue in R | )
Continuous: Color each element
1/2 red and 1/2 blue (0 discrepancy)

Discrete:
Random has about O(n
1/2
log
1/2
n)
Can achieve O(log
2.5
n)
9/30
Combinatorial Discrepancy
Universe: U= [1,,n]
Subsets: S
1
,S
2
,,S
m


Color elements red/blue so each
set is colored as evenly as possible.

Find _: [n] ! {-1,+1} to
Minimize |_(S)|
1
= max
S
|
i 2 S
_(i) |

If A is m \times n incidence matrix.
Disc(A) = min_{x \in {-1,1}^n} |Ax|_\infty

S
1
S
2
S
3
S
4
10/30
Applications
CS: Computational Geometry, Comb. Optimization, Monte-Carlo
simulation, Machine learning, Complexity, Pseudo-Randomness,

Math: Dynamical Systems, Combinatorics, Mathematical Finance,
Number Theory, Ramsey Theory, Algebra, Measure Theory,

Hereditary Discrepancy
11/30
Rounding
Lovasz-Spencer-Vesztermgombi86
Given any matrix A, and x \in R^n
can round x to \tilde{x} \in Z^n s.t.
|Ax A\tilde{x}|_\infty < Herdisc(A)

Proof: Round the bits one by one.
12/30
Can we find it efficiently?

Nothing known until recently.

Thm [B10]. Can efficiently round so that
Error \leq O(\sqrt{log m log n})
Herdisc(A)
13/30
More rounding approaches
Bin Packing

Refined further by Rothvoss (Entropy rounding
method)
14/30
Dynamic Data Structures
N points in a 2-d region.
Weights update over time.

Query: Given an axis-parallel rectangle R,
determine the total weight on points in R.
Preprocess:
1) Low query time
2) Low update time (upon weight change)
15/30
Example
Line:
Query = O(n) Update = 1

Query = 1 Update = O(n^2)
Query = 2 Update = O(n)

Query = O(log n) Update = O(log n)

Recursively can get for 2-d.

16/30
What about other objects?
Query


Circles arbitrary rectangles aligned triangle

Turns out t_q t_u \geq n^{1/2}/log^2 n ?
Larsen-Green: t_q t_u \geq disc(S)^n/log^2 n
17/30
Sketch of idea


A good data structure implies
D = A P


A = row sparse P = Column sparse
(low query time) (low update time)
18/30
Outline again


19/30
Basic Results


20/30
21/30
Best Known Algorithm
Random: Color each element i independently as
x(i) = +1 or -1 with probability each.

Thm: Discrepancy = O (n log n)
1/2


Pf: For each set, expect O(n
1/2
) discrepancy
Standard tail bounds: Pr[ |
i 2 S
x(i) | c n
1/2
] e
-c
2

Union bound + Choose c (log n)
1/2

Analysis tight: Random actually incurs O((n log n)
1/2
).

22/30
Better Colorings Exist!
[Spencer 85]: (Six standard deviations suffice)
Always exists coloring with discrepancy 6n
1/2

(In general for arbitrary m, discrepancy = O(n
1/2
log(m/n)
1/2
)
Tight: For m=n, cannot beat 0.5 n
1/2
(Hadamard Matrix, orthogonal sets)

Inherently non-constructive proof
(pigeonhole principle on exponentially large universe)

Challenge: Can we find it algorithmically ?
Certain algorithms do not work [Spencer]

Conjecture [Alon-Spencer]: May not be possible.

23/30
Beck Fiala Thm
U = [1,,n] Sets: S
1
,S
2
,,S
m


Suppose each element lies in at most t sets (t << n).

[Beck Fiala 81]: Discrepancy 2t -1.
(elegant linear algebraic argument, algorithmic result)

Beck Fiala Conjecture: O(t
1/2
) discrepancy possible

Other results: O( t
1/2
log t log n ) [Beck]
O( t
1/2
log n ) [Srinivasan]
O( t
1/2
log
1/2
n ) [Banaszczyk]
S
1
S
2
S
3
S
4

Non-constructive
24/30
Approximating Discrepancy
Question: If a set system has low discrepancy (say << n
1/2
)
Can we find a good discrepancy coloring ?

[Charikar, Newman, Nikolov 11]:
Even 0 vs. O (n
1/2
) is NP-Hard



(Matousek): What if system has low Hereditary discrepancy?
herdisc (U,S) = max
U U
disc (U, S
|U
)

Robust measure of discrepancy (often same as discrepancy)
Widely used: TU set systems, Geomety,
S
1

S
2


S
1

S
2


1 2 n 1 2 n
25/30
Our Results
Thm 1: Can get Spencers bound constructively.
That is, O(n
1/2
) discrepancy for m=n sets.

Thm 2: If each element lies in at most t sets, get bound of
O(t
1/2
log n) constructively (Srinivasans bound)

Thm 3: For any set system, can find
Discrepancy O(log (mn)) Hereditary discrepancy.
Other Problems: Constructive bounds (matching current best)
k-permutation problem [Spencer, Srinivasan,Tetali]
Geometric problems ,
26/30
Relaxations: LPs and SDPs
Not clear how to use.

Linear Program is useless. Can color each element red and
blue. Discrepancy of each set = 0!

SDPs (LP on v
i
v
j
, cannot control dimension of vs)

|
i 2 S
v
i
|
2
n 8 S
|v
i
|
2
= 1
Intended solution v
i
= (+1,0,,0) or (-1,0,,0).
Trivially feasible: v
i
= e
i
(all v
i
s orthogonal)
Yet, SDPs will be a major tool.
27/30
Punch line
SDP very helpful if tighter bounds needed for some sets.

|
i 2 S
v
i
|
2
2 n
|
i 2 S
v
i
|
2
n/log n
|v
i
|
2
1

Not apriori clear why one can do this.
Entropy Method.

Algorithm will construct coloring over time and
use several SDPs in the process.

Tighter bound for S
28/30
Talk Outline
Introduction

The Method
Low Hereditary discrepancy -> Good coloring

Additional Ideas
Spencers O(n
1/2
) bound


Partial Coloring Method
29/30
A Question
30/30
-n n
Slight improvement
Can be improved to O(\sqrt{n})/2^n

If you pick a random {-1,1} coloring s
w.p. say >= |a \cdot s| \leq c \sqrt{n}

2^{n-1} colorings s, with |a\cdot s| \leq c
\sqrt{n}

31/30
Algorithmically
Easy: 1/poly(n) (How?)
Answer: Pick any poly(n) colorings.

[Karmarkar-Karp81]: \approx 1/n^log n

Huge gap: Major open question

Remark: {-1,+1} not enough. Really need
color 0 also.
E.g. a_1 = 1, a_2==a_n = 1/(2n)

32/30
Yet another enhancement
There is a {-1,0,1} coloring with at least
n/2 {-1,1}s s.t. \sum_i a_i s_i \leq n/2^{n/5}

Make buckets of size 2n/2^{n/5}
At least 2^{4n/5} sums fall in same bucket

Claim: Some two s and s in same bucket and differ in at
least n/2 coordinates

Again consider s = (s-s)/2
33/30
Proof of Claim
Claim: Any set of 2^{4n/5} vertices of the
boolean cube has

[Kleitman66] Isoperimetry for cube.
Hamming ball B(v,r) has the smallest
diameter for a given number of vertices.

|B(v,n/4)| < 2^{4n/5}
34/30
Spencers proof

35/30
36/30
Our Approach
37/30
Algorithm (at high level)
Cube: {-1,+1}
n
Analysis: Few steps to reach a vertex (walk has high variance)
Disc(S
i
) does a random walk (with low variance)
start
finish
Algorithm: Sticky random walk
Each step generated by rounding a suitable SDP
Move in various dimensions correlated, e.g. o
t
1
+ o
t
2
0
Each dimension: An Element
Each vertex: A Coloring
38/30
An SDP
Hereditary disc. ) the following SDP is feasible
SDP:
Low discrepancy: |
i 2 S
j
v
i
|
2

2

|v
i
|
2
= 1
Rounding:
Pick random Gaussian g = (g
1
,g
2
,,g
n
)
each coordinate g
i
is iid N(0,1)

For each i, consider q
i
= g v
i
Obtain v
i
2 R
n

39/30
Properties of Rounding
Lemma: If g 2 R
n
is random Gaussian. For any v 2 R
n
,
g v is distributed as N(0, |v|
2
)
Pf: N(0,a
2
) + N(0,b
2
) = N(0,a
2
+b
2
) g v =
i
v(i) g
i
N(0,
i
v(i)
2
)

1. Each q
i
N(0,1)
2. For each set S,

i 2 S
q
i
= g (
i2 S
v
i
) N(0,
2
)
(std deviation )
SDP:
|v
i
|
2
= 1
|
i2

S
v
i
|
2

2

Recall: q
i
= g v
i
qs mimics a low discrepancy coloring (but is not {-1,+1})
40/30
Algorithm Overview
Construct coloring iteratively.
Initially: Start with coloring x
0
= (0,0,0, ,0) at t = 0.
At Time t: Update coloring as x
t
= x
t-1
+ (q
t
1
,,q
t
n
)
( tiny: 1/n suffices)

x(i)
x
t
(i) = (q
1
i
+ q
2
i
+ + q
t
i
)

Color of element i: Does random walk
over time with step size N(0,1)
Fixed if reaches -1 or +1.
time
-1
+1
Set S: x
t
(S) =
i 2 S
x
t
(i) does a random walk w/ step N(0,
2
)

41/30
Analysis

Consider time T = O(1/
2
)

Claim 1: With prob. , at least n/2 elements reach -1 or +1.
Pf: Each element doing random walk with size .
Recall: Random walk with step 1, is O(t
1/2
) away in t steps.

A Trouble: Various element updates are correlated

Consider basic walk x(t+1) = x(t) 1 with prob
Define Energy u(t) = x(t)
2

E[u(t+1)] = (x(t)+1)
2
+ (x(t)-1)
2
= x(t)
2
+ 1 = u(t)+1
Expected energy = n at t= n.

Claim 2: Each set has O() discrepancy in expectation.
Pf: For each S, x
t
(S) doing random walk with step size
42/30
Analysis

Consider time T = O(1/
2
)
Claim 1: With prob. , at least n/2 variables reach -1 or +1.

) Everything colored in O(log n) rounds.

Claim 2: Each set has O() discrepancy in expectation per round.

) Expected discrepancy of a set at end = O( log n)


Thm: Obtain a coloring with discrepancy O( log (mn))
Pf: By Chernoff, Prob. that disc(S) >= 2 Expectation + O( log m)
= O( log (mn))
is tiny (poly(1/m)).
43/30
Recap
At each step of walk, formulate SDP on unfixed variables.
Use some (existential) property to argue SDP is feasible
Rounding SDP solution -> Step of walk

Properties of walk:
High Variance -> Quick convergence
Low variance for discrepancy on sets -> Low discrepancy
44/30
Refinements
Spencers six std deviations result:
Goal: Obtain O(n
1/2
) discrepancy for any set system on m = O(n) sets.

Random coloring has n
1/2
(log n)
1/2
discrepancy


Previous approach seems useless:
Expected discrepancy for a set O(n
1/2
),
but some random walks will deviate by up to (log n)
1/2
factor

Need an additional idea to prevent this.
45/30
Spencers O(n
1/2
) result

Partial Coloring Lemma: For any system with m sets, there exists a
coloring on n/2 elements with discrepancy O(n
1/2
log
1/2
(2m/n))
[For m=n, disc = O(n
1/2
)]


Algorithm for total coloring:
Repeatedly apply partial coloring lemma
Total discrepancy
O( n
1/2
log
1/2
2 ) [Phase 1]
+ O( (n/2)
1/2
log
1/2
4 ) [Phase 2]
+ O((n/4)
1/2
log
1/2
8 ) [Phase 3]
+ = O(n
1/2
)
46/30
Proving Partial Coloring Lemma
Beautiful Counting argument (entropy method + pigeonhole)
Idea: Too many colorings (2
n
), but few discrepancy profiles

Key Lemma: There exist k=2
4n/5
colorings X
1
,,X
k
such that
every two X
i
, X
j
are similar for every set S
1
,,S
n
.


Some X
1
,X
2
differ on n/2 positions
Consider X = (X
1
X
2
)/2


Pf: X(S) = (X
1
(S) X
2
(S))/2 2 [-10 n
1/2
, 10 n
1/2
]
X
1
= ( 1,-1, 1 , ,1,-1,-1)
X
2
= (-1,-1,-1, ,1, 1, 1)
X = ( 1, 0, 1 , ,0,-1,-1)
47/30
A useful generalization
There exists a partial coloring with non-uniform
discrepancy bound A
S
for set S

Even if A
S
= O( n
1/2
) in some average sense
48/30
An SDP
Suppose there exists partial coloring X:
1. On n/2 elements
2. Each set S has |X(S)| A
S

SDP:
Low discrepancy: |
i 2 S
j
v
i
|
2
A
S
2

Many colors:
i
|v
i
|
2
n/2
|v
i
|
2
1
Pick random Gaussian g = (g
1
,g
2
,,g
n
)
each coordinate g
i
is iid N(0,1)

For each i, consider q
i
= g v
i
Obtain v
i
2 R
n


49/30
Algorithm
Initially write SDP with A
S
= c n
1/2


Each set S does random walk and expects to reach
discrepancy of O(A
S
) = O(n
1/2
)

Some sets will become problematic.
Reduce their A
S
on the fly.
Not many problematic sets, and entropy penalty low.


0
20n
1/2
30n
1/2
35n
1/2
Danger 1 Danger 2 Danger 3
50/30
Concluding Remarks
Construct coloring over time by solving sequence of SDPs
(guided by existence results)

Works quite generally

Can be derandomized [Bansal-Spencer]
(use entropy method itself for derandomizing + usual tech.)
E.g. Deterministic six standard deviations can be viewed as a way to
derandomize something stronger than Chernoff bounds.
51/30
Thank You!
52/30

53/30

54/30
Rest of the talk
1. How to generate q
i
with required properties.
2. How to update A
S
over time.
Show n
1/2
(log log log n)
1/2
bound.
55/30
Why so few algorithms?
Often algorithms rely on continuous relaxations.
Linear Program is useless. Can color each element
red and blue.

Improved results of Spencer, Beck, Srinivasan,
based on clever counting (entropy method).
Pigeonhole Principle on exponentially large systems
(seems inherently non-constructive)

56/30
Partial Coloring Lemma
Suppose we have discrepancy bound A
S
for set S.





Consider 2
n
possible colorings
Signature of a coloring X: (b(S
1
), b(S
2
),, b(S
m
))

Want partial coloring with signature (0,0,0,,0)
57/30
Progress Condition
Energy increases at each step:
E(t) = \sum_i x_i(t)^2
Initially energy =0, can be at most n.

Expected value of E(t) = E(t-1) + \sum_i
\gamma_i(t)^2
Markovs inequality.
58/30
Missing Steps
1. How to generate the \eta_i
2. How to update \Delta_S over time
59/30
Partial Coloring
If exist two colorings X
1
,X
2

1. Same signature (b
1
,b
2
,,b
m
)
2. Differ in at least n/2 positions.

Consider X = (X
1
X
2
)/2
1. -1 or 1 on at least n/2 positions, i.e. partial coloring
2. Has signature (0,0,0,,0)
X(S) = (X
1
(S) X
2
(S)) / 2, so |X(S)| A
S
for all S.

Can show that there are 2
4n/5
colorings with same signature.
So, some two will differ on > n/2 positions. (Pigeon Hole)
X
1
= (1,-1, 1 , , 1,-1,-1)
X
2
= (-1,-1,-1, , 1,1, 1)
60/30

61/30
Spencers O(n
1/2
) result

Partial Coloring Lemma: For any system with m sets,
there exists a coloring on n/2 elements with
discrepancy O(n
1/2
log
1/2
(2m/n))
[For m=n, disc = O(n
1/2
)]

Algorithm for total coloring:
Repeatedly apply partial coloring lemma
Total discrepancy
O( n
1/2
log
1/2
2 ) [Phase 1]
+ O( (n/2)
1/2
log
1/2
4 ) [Phase 2]
+ O((n/4)
1/2
log
1/2
8 ) [Phase 3]
+ = O(n
1/2
)
Let us prove the lemma for m = n
62/30
Proving Partial Coloring Lemma
Pf: Associate with coloring X, signature = (b
1
,b
2
,,b
n
)
(b
i
= bucket in which X(S
i
) lies )
Wish to show: There exist 2
4n/5
colorings with same signature

Choose X randomly: Induces distribution on signatures.
Entropy () n/5 implies some signature has prob. 2
-n/5
.

Entropy ( )
i
Entropy( b
i
) [Subadditivity of Entropy]

b
i
= 0 w.p. 1- 2 e
-50
,
= 1 w.p. e
-50

= 2 w.p. e
-450

.
-10 n
1/2
-30 n
1/2
10 n
1/2
30 n
1/2
0 2 1 -1 -2
Ent(b
1
) 1/5
63/30
A useful generalization
Partial coloring with non-uniform discrepancy A
S
for set S
A
S
3A
S
A
S
3A
S
5A
S
0 1 2
-1 -2
For each set S, consider the bucketing
Suffices to have
s
Ent (b
s
) n/5

Or, if A
S
=
s
n
1/2
, then
s
g(
s
) n/5

g() e
-
2
/2
> 1
ln(1/) < 1
Bucket of n
1/2
/100
has penalty ln(100)
64/30
Recap
Partial Coloring: A
S
10 n
1/2
gives low entropy
) 2
4n/5
colorings exist with same signature.
) some X
1
,X
2
with large hamming distance.
(X
1
X2) /2 gives the desired partial coloring.






Trouble: 2
4n/5
/2
n
is an exponentially small fraction.
Only if we could find the partial coloring
efficiently