1.

Polynomials
Real polynomial p in one variable x:
p(x) = anxn + an-1xn-1 + ... +a1x1 + a0

Algorithms Theory

a0 ,..., an R, an 0: coefficients of p
degree of p: highest power of x in p (= n)

02 - Polynomial Multiplication and

Fast Fourier Transform

Example:
p(x) = 3x3 15x2 + 18x

Prof. Dr. S. Albers

Set of all real polynomials: R[x]

Winter term 07/08

Winter term 07/08

2. Operations on polynomials

Operations on polynomials

p, q R[ x]

2. Multiplication:

p( x )q( x ) = (an x n + K + a0 )(bn x n + K + b0 )

= c2 n x 2 n + K + c1 x1 + c0

p( x ) = an x n + K + a1 x1 + a0
q( x ) = bn x n + K + b1 x1 + b0

c i : What products of monomials have degree i ?

1. Addition

ci =

j =0

i = 0,...,2n.

an+1 = K = a2 n = 0, bn+1 = K = b2 n = 0

p( x ) + q( x ) = (an x n + K + a0 ) + (bn x n + K + b0 )
= (an + bn )x n + K + (a1 + b1 )x1 + (a0 + b0 )

Winter term 07/08

a j bi j

Polynomial ring R[x].

Winter term 07/08

Operations on polynomials

3. Representation of polynomials
p(x) R[x]

3. Evaluation at a specific point x0: Horners method

Possible representations of p(x):

p( x0 ) = (K(an x0 + an1 )x0 + K + a1 )x0 + a0

1. Coefficient representation
Running time: O(n)

p( x ) = an x n + K + a1 x1 + a0
Example:

p( x ) = 3 x 3 15 x 2 + 18 x

Winter term 07/08

Representation of polynomials

Winter term 07/08

Representation of polynomials

2. Product of linear factors

3. Point-value representation

p(x) R[x]

Interpolation lemma:
Any polynomial p(x) R[x] of degree n is uniquely defined by
n+1 pairs (xi, p(xi)), where i = 0,...,n and xi xj for i j.

p( x ) = an ( x x1 )K( x xn )

Example:
The polynomial

Example:

p( x ) = 3 x( x 2 )( x 3)

p( x ) = 3x( x 2 )( x 3)

is uniquely defined by the point-value pairs (0,0), (1,6), (2,0), (3,0).

Winter term 07/08

Winter term 07/08

Operations on polynomials

Operations on polynomials

p, q R[x], degree(p) = degree(q) = n

Addition:

p + q = ( x0 , y0 + z0 ), ( x1 , y1 + z1 ),K, ( xn , yn + zn )

Coefficient representation
Addition:
O(n)
Multiplication:
O(n2)
Evaluation at x0 : O(n)

Running time: O(n)

Multiplication:

p q = ( x0 , y0 z0 ), ( x1 , y1 z1 ),K, ( xn , yn zn )
(Condition: n degree(pq))

Point-value representation

p = ( x0 , y0 ), ( x1 , y1 ),K, ( xn , yn )

Running time: O(n)

q = ( x0 , z0 ), ( x1 , z1 ),K, ( xn , zn )

Evaluation at point x: ??
Convert polynomial to coefficient representation
(interpolation)

Winter term 07/08

Polynomial multiplication

Winter term 07/08

10

Divide-and-conquer approach
Idea: (assume n is even)

p( x ) = a0 + a1 x + K + an 1 x n 1

Compute the product of two polynomials p, q of degree < n:

= a0 + a 2 x 2 + K + a n 2 x n 2 +

p,q of degree n-1, n coefficients

Evaluation:
2n point-value pairs

( x , p( x ))
i

x0 , x1 , K , x2 n 1
und

a1 x + a3 x 3 + K + an 1 x n 1

( x , q( x ))
i

Pointwise multiplication

x a1 + a3 x 2 + K + an 1

2n point-value pairs ( x , pq( x ))

i

( )(
(x )(

= a0 + a 2 x 2 + K + a n 2 x 2

( )

( )

n2 )/ 2

n2 )/ 2

= p0 x 2 + xp1 x 2

Interpolation
pq of degree 2n-2, 2n-1 coefficients

Winter term 07/08

11

Winter term 07/08

12

Representation of p(x)

Discrete Fourier Transform

Assume: degree(p) < n

The values p(ni) of the n powers of n uniquely define p if degree(p) < n.

3a. Values of the n powers of the principal nth root of unity

n = e2i / n
i = 1

Discrete Fourier Transform (DFT)

e2i = 1

83

Powers of n (roots of unity):

1=

n , n ,K, n
0

DFTn ( p ) = ( p( n0 ), p( n1 ),K, p( nn 1 ))

n 1

Example: n = 4

81

84

eix = cos x + i sin x

40 = e0i = cos(0) + i sin(0) = 1

80 = 1
85

86

Winter term 07/08

41 = e 2i / 4 = cos( / 2) + i sin( / 2) = i
42 = (e2i / 4 ) = cos + i sin = 1
2

87

43 = (e 2i / 4 ) = cos(3 / 2 ) + i sin (3 / 2 ) = i
3

13

Evaluation at the roots of unity

Winter term 07/08

14

Evaluation at the roots of unity

p ( x ) = 3 x 3 15 x 2 + 18 x

41 = i

42 = 1

( , p( )) = (1, p(1)) = (1,6)

( , p( )) = (i, p(i )) = (i, 15 + 15i )
( , p( )) = (-1, p(-1)) = (-1, -36)
( , p( )) = (-i, p(-i)) = (-i, 15 -15i )

40 = 1

43 = i

Winter term 07/08

DFT4 ( p) = (6,15 + 15i,36,15 15i )

15

Winter term 07/08

16

Polynomial multiplication

4. Properties of the roots of unity

Compute the product of two polynomials p, q of degree < n:

20n , 21n ,K, 22nn1 form a multiplicative group

p, q of degree n-1, n coefficients

Cancellation lemma:
For any integers n > 0, k 0 and d > 0 we have:

Evaluation:
2n point-value pairs

i
2n

20n , 21n ,K, 22nn1

, p( 2i n )) and ( 2i n , q( 2i n ))

dndk = nk

Pointwise multiplication
2n point-value pairs

Proof:

, pq( 2i n ))
2n

dndk = e 2idk / ( dn ) = e 2ik / n = nk

Interpolation
pq of degree 2n-2, 2n-1 coefficients
Therefore:

Winter term 07/08

17

5. Discrete Fourier Transform

2nn = 21 = 1

Winter term 07/08

18

Discrete Fourier Transform

Idea: (assume n is even)

DFTn ( p ) = ( p( n0 ), p( n1 ),K, p( nn 1 ))

p( x ) = a0 + a1 x + K + an 1 x n 1
= a0 + a 2 x 2 + K + a n 2 x n 2 +

Fast Fourier Transform:

a1 x + a3 x 3 + K + an 1 x n 1

( )(
(x )(

= a0 + a 2 x 2 + K + a n 2 x 2

Computation of DFTn(p) by means of a divide-and-conquer approach.

x a1 + a3 x 2 + K + an 1

( )

( )

= p0 x + xp1 x
2

n 2 )/ 2

n 2 )/ 2

p0 ( x ) = a0 + a2 x + K + an 2 x ( n 2 ) / 2
p1 ( x ) = a1 + a3 x + K + an1 x ( n 2 ) / 2
Winter term 07/08

19

Winter term 07/08

20

Discrete Fourier Transform

Discrete Fourier Transform

Example:

Evaluation for k = 0, ... , n 1:

p( nk ) = p0 (( nk ) ) + nk p1 (( nk )
2

p0 nk / 2

if

=
k n / 2
p0 n / 2

if

) + p ( ),

p ( 40 ) = p0 ( 20 ) + 40 p1 ( 20 )

k < n/2
+ nk p1 nk /2n / 2 ,

p ( 41 ) = p0 ( 21 ) + 41 p1 ( 21 )

k
n

k
n/2

p ( 42 ) = p0 ( 20 ) + 42 p1 ( 20 )

k n/2

Winter term 07/08

p ( 43 ) = p0 ( 21 ) + 43 p1 ( 21 )

21

Computation of DFTn

22

A further improvement

DFT ( p ) = ( p( ), p( ),K, p(
n

Winter term 07/08

n 1

))

( )

Base case: n = 1 (degree(p) = n 1 = 0)

DFT1(p) = a0
General case :
Divide:
Divide p into p0 and p1
Conquer:
Recursively compute DFTn/2(p0) and DFTn/2(p1).
Merge:

p0 nk/ 2 + nk p1 nk/ 2

p =
p0 nk/2n / 2 + nk p1 nk/2n / 2
k
n

if k < n/ 2

p0 nk / 2 + nk p1 nk/ 2

=
p0 nk/2n / 2 nk n / 2 p1 nk /2n / 2

if k n / 2

if k < n/ 2

if k n / 2

For k = 0, ... , n 1 compute:

DFTn(p)k =

( DFTn / 2 ( p0 ), DFTn / 2 ( p0 )) k +

Thus, if k < n/2:

n ( DFTn / 2 ( p1 ), DFTn / 2 ( p1 )) k
k

Winter term 07/08

23

Winter term 07/08

p0 ( nk/ 2 ) + nk p1 ( nk/ 2 ) = p( nk )
p0 ( nk/ 2 ) nk p1 ( nk/ 2 ) = p( nk + n / 2 )
24

A further improvement

6. Fast Fourier Transform

Example:

Algorithm: FFT
Input: Array a containing the n coefficients of a polynomial p and n = 2k
Output: DFTn(p)

p ( 40 ) = p0 ( 20 ) + 40 p1 ( 20 )

1.
2.
3.
4.

p ( 41 ) = p0 ( 21 ) + 41 p1 ( 21 )
p ( 42 ) = p0 ( 20 ) 40 p1 ( 20 )
p ( 43 ) = p0 ( 21 ) 41 p1 ( 21 )

n = e2i/n
6. = 1
5.

7.
8.
9.
10.
11.
Winter term 07/08

25

FFT: Example

for k = 0 to n/2 1 do
d k = d k[0 ] + d k[1]
d k + n / 2 = d k[0 ] d k[1]
= n

/* = nk*/

return d

Winter term 07/08

26

7. Analysis
p( x ) = 3 x 3 15 x 2 + 18 x + 0

T(n) = Time required for evaluating a polynomial of degree < n at

0
1
2 n 1
the points 2 n , 2 n ,K, 2 n .

a = [0, 18, -15, 3 ]

a[0] = [0, -15]
a[1] = [18, 3]
FFT([0, -15], 2) = (FFT([0],1) + FFT([-15],1), FFT([0],1) - FFT([-15],1))
= (-15,15)
FFT([18, 3],2) = (FFT([18],1) + FFT([3],1), FFT([18],1) - FFT([3],1))
= (21,15)
k=0; =1
d0 = -15 + 1 * 21 = 6

d2 = -15 1 * 21 = -36

k=1;=i
d1 = 15 + i*15

d3 = 15 i*15

FFT(a, 4)

if n = 1 then /* p is constant */
return a
d[0] = FFT([a0, a2, ... , an-2 ], n/2)
d[1] = FFT([a1, a3, ... , an-1], n/2)

T(1) = O(1)
T(n) = 2 T(n/2) + O(n)
= O(n log n)

= (6, 15+15i, -36, 15 15i)

Winter term 07/08

27

Winter term 07/08

28

Polynomial multiplication

Interpolation

Compute the product of two polynomials p, q of degree < n:

Converte the point-value representation into coefficient representation.

p,q of degree n-1, n coefficients

Input:

Evaluation via FFT:

2n point-value pairs

i
2n

20n , 21n , K, 22nn 1

, p( 2i n )) and ( 2i n , q( 2i n ))

(x0, y0),..., (xn-1, yn-1) where xi xj, for all i j

Output: Polynomial p with coefficients a0,..., an-1,

such that

Pointwise multiplication
2n point-value pairs

i
2n

, pq( 2i n ))

p( x0 ) = a0 + a1 x0 + K + an1 x0n 1 = y0
p( x1 ) = a0 + a1 x1 + K + an1 x1n1 = y1
p( x2 ) = a0 + a1 x2 + K + an1 x2n1 = y2
M
M
p( xn1 ) = a0 + a1 xn1 + K + an1 xnn11 = yn1

Interpolation
pq of degree 2n-2, 2n-1 coefficients

Winter term 07/08

29

Interpolation

Winter term 07/08

30

Interpolation

Matrix notation:

System of equations
1 x0 L x0n 1 a0 y0

1 x1 L x1n 1 a1 y1

M = M
M

n 1
1 x
a yn 1
x
L
n 1
n 1 n 1

solvable if xi x j for all i j.

n 1
1 x0 L x0 a0 y0

n 1
1 x1 L x2 a1 y1
M = M

n 1
1 xn1 L xn1 an1 yn1

Special case (here) :

xi = ni

Definition:

Vn = ( nij )i , j , a = (ai ),
Vn a = y

Winter term 07/08

31

Winter term 07/08

y = ( yi )

a = Vn1 y
32

Interpolation

Interpolation
Consider the entry of Vn1Vn in line i and column j :

Theorem:
For any 0 i, j n 1 we have:

(V )
1

Proof:

ij

Vn1 =
n

1
n
n ij

n
ij
n

We have to show:

(V

n ij

1
n

i , j

i
n

L
L

i ( n 1)
n

M
L

33

Interpolation

Winter term 07/08

34

Interpolation

(V

Vn )ij =

n 1

n ik

k =0

njk =

Summation lemma:
For any integer n > 0, l 0 with n

1 n1 ( i + j ) k
n
n k =0

n 1

l:

n 1

Case 1: i = j

k =0

n 1

1
1
n(i + j )k = n0k = 1
n
n k =0
k =0

lk
n

=0

Proof:

( ) 1 = ( ) 1 = 0
(
)

1
1

i.e. (n 1) i + j n 1
thus n /| i + j :

n 1

k =0

1 n 1 (i + j )k
n
=0
n
k =0

Winter term 07/08

L
L

Lij

Vn1Vn = I n

Winter term 07/08

Case 2: i j,

L
1

L j
n

2j

L n( n 1) j

35

Winter term 07/08

l k
n

l n
n
l
n

n l
n
l
n

36

Interpolation

Interpolation

ai = Vn1 y

y0

1 ni
ni (n 1) y1

,K,
= ,
n M
n n

y
n 1
n 1

= yk
k =0

a=

a=

ik
n

n 1

n 1

1
(r ( n0 ), r ( n1 ),K, r ( n( n1) ))
n

( )

1 n 1
yk ni

n k =0

Interpolation and DFT

(( ) (

Winter term 07/08

38

Polynomial multiplication by FFT

Compute the product of two polynomials p, q of degree < n:

1
a = (r ( n0 ), r ( n1 ),K, r ( n( n1) ))
n

( ))

1
r nn , r nn 1 , K , r n1
n

1
( DFTn (r )) n i
n
1
a0 = ( DFTn (r )) 0
n
Winter term 07/08

yk ( n0 ) , k=0 yk ( n1 ) ,L, k=0 yk ( n( n1) )

k =0
n 1

37

ai =

r ( x ) = y0 + y1 x + y2 x 2 + L + yn1 x n1

Winter term 07/08

a=

1
n

p,q of degree n-1, n coefficients

Evaluation by FFT:

since

nn = 1

2n point-value pairs

i
2n

20n , 21n , K, 22nn 1

, p( 2i n )) und ( 2i n , q( 2i n ))

Pointwise multiplication
2n point-value pairs

(i 0)

i
2n

, pq( 2i n ))

Interpolation via FFT

pq of degree 2n-2, 2n-1 coefficients

39

Winter term 07/08

40