Fourier transforms

http://aurora.phys.utk.edu/~forrest/papers/fourier/index.html
http://www.yorvic.york.ac.uk/~cowtan/fourier/fourier.html
DFTs:
http://www.spd.eee.strath.ac.uk/~interact/fourier/fft.html,
http://www.4p8.com/eric.brasseur/fouren.html,
FFTs:
http://www.dspguide.com/pdfbook.htm, Chapter 12
Wavelets:
http://www.amara.com/IEEEwave/IEEEwavelet.html#contents
 Basic Fourier transform
• Multiply function by e-2π isx and integrate
• But e-2π isx is cos(2π sx) - i sin(2π sx)
– result is to multiply function by sin or cos and integrate
Signal wave Signal wave

Sine waves Products of sine and signal

 Sample - square step signal

Note that sine wave has positive and negative values

•
Some components integrate to zero
freq
•
Lower frequencies contribute more
• 1

Fourier component amplitudes 3

products
4
amplitude

of sine
& signal

Positive part 5
dc 2 4
frequency Negative part
 Sample -- time-limited sine wave
• Common in holography
• All signals have some time limit
Signal (8 cycles of sine wave)
Freq of sine wave
component
1
Product waveforms
signal times sine

1.05

1.1
Zero integral for
1.15
deviation ~ 1/#cycles

1.2

1.25
 • Which is better
Sine vs cosine
– depends on choice of origin
– symmetric function -- cosine, anti-symmetric -- sine
• Really want cos(2π sx + φ ) or sin(2π sx + φ )
– φ contains origin info
• Can be built up from sin and cos
– A sin(2π sx) + B cos(2π sx) = √(A2+B2) cos(2π sx+tan-1 (B/A)) Phase
shift of sine
component
Signal (8 cycles of sine wave)
0
Product waveforms
-- signal times sine

0.2π

0.4π
0.5π = cosine
0.6π

0.8π
π
 Most important property of
Fourier transforms
• Fourier transform of product of 2 functions
– convolution of Fourier transforms
• F.T. of convolution is product (and converse)
• great simplification -- critical for many applications

Product Convolution

Signal 1 Transform 1

Signal 2
Transform 2
 Discrete Fourier transform
• Fixed time sampling intervals
– not continuous
• Sampling interval must be fine enough
– aliasing
– Nyquist limit t = 1/(2f), where f is maximum Fourier frequency in signal

Coarse sampling Fine sampling

 Aliasing
• Time sampling too far apart
• Looks like sine wave of different freq
Over-sampled -- faithful representation

Under-sampled (solid lines)

 More examples of aliasing
• Theory developed for sound reproduction
– digital music CDs
 Fast Fourier Transform (FFT)
• Normal DFT takes N operations
2

• FFT takes NlogN operations

– very important as N grows larger
– many key DFT applications would not be practical without it
 Form of FFT
• Use complex representation
– reduces second order diff eqns to first order eqns
• Lowest freq component W = e2π i/N
= cos(2π /N) + i sin(2π /N)
– exactly one cycle over full waveform
• Use powers of lowest freq

N −1 As in DFT, integral replaced by sum

Fω = ∑ W ωt f t Length of time step omitted
• usually constant
t =0
time
frequency
Real DFT vs complex DFT

cosine

sine
 Decomposition step
Decimation
• Each step reduces size of problem by 2
• creates twice as many problems
– each problem simpler
– original problem N2 computation steps
– reduced problem N single computation steps
– log2N steps to reduce
– result Nlog2N vs N2
–

even odd

ee eo oe oo
 Decomposition step example
• Each step reduces waveform complexity
• Final result is single sample point
• Fourier transform of single value = the value

0 15
even odd

ee eo oe oo
 •
•

–
0000
0000 1000 0000 0000 0000
1000 0100 0010 0001
0100 1000 0100
0100 0010
1100 0110
1100 1100 0011
0010 1000 0100
0010 1010
1010 0101
like perfect un-shuffle operation

0110 1100
0110 1010 1110 0110
1110
Each step sorts by least significant bit

1110 0111
0001 1000
1001 0001 1001
Look at binary representations of sample indices

0101 0011 1010

1101 0101 1011
0111
0011 1100
1001
1011 1011 1101
0111 1101 1110
1111 1111 1111
Decomposition is simple swap

0110
 Fourier transform step
• FT of single-valued function is itself
• Do nothing

N −1 0
Fω = ∑ W f t ωt N=1
F0 = ∑ W 0t f t = f 0
t =0 t =0
time
frequency

W = e2π i/N
 Recombination step
• Reconstruct original function by reversing decomposition steps
• Must work in Fourier domain

Expand by padding with zeros

Time shift odd part

Sum
 Summing step in recombination
• FT of sum = sum of FTs

(f ) = ∑W
N −1 N −1 N −1
Fω = ∑ W ωt
t
(1)
+ ft ( 2) ωt
f t + ∑ W ωt f t ( 2 ) = Fω(1) + Fω( 2 )
(1)

t =0 t =0 t =0

time
frequency

W = e2π i/N

Sum
 Time-shift step in recombination
• Multiply transform by “sine wave” -- W ω = e 2πiω / N
– Product of FT and sine = convolution of function with time-shifted delta function

N −1 N −2 N −1
Fω = ∑ W ωt f t = ∑ ft +
W ωt
∑ ft
W ωt

t =0 t = 0 , 2 ,... t =1, 3,...

frequency time Time shifter
N / 2 −1 N / 2 −1

2t’ = t
= ∑ W ω 2t ' e

t '= 0
ft ' + ∑ W ω ( 2 t ' +1) o

t '= 0
f t' = Fω
e
+ W ω o
Fω

Time shift odd part

 Expand by padding with zeros
Non-intuitive
• Expand by dividing frequency by 2
• does not change number of points
–
To add points need to add FT freq. Components (Nyquist f=1/2N)
• What to add?
• Frequencies from N to 2N-1
–

Freq. from 0 to N
N −1 N −1
Fω = ∑ W f t ωt expand
Fω / 2 = ∑ W ( ω / 2 ) t f t
t =0 t =0
frequency time
Add frequencies

Fω = Fω / 2 + FN +ω / 2
Expand by padding with zeros
Choose so that f odd = 0
Example - expand and pad with zeros
• Divide frequency by 2
• Add high frequency Fourier component
• Same amplitude

cos(2π x/4) Pad with zeros cos(2π x/8) + cos(2π x7/8)

 Example - expand, pad, time-shift
• Correct result

cos(2π x/4) Pad with zeros cos(2π x/8) + cos(2π x7/8)

Time shift
sin(2π x/8) [cos(2π x/8) + cos(2π x7/8)]
 Operations needed - pad with
zeros and time-shift
• Duplicate low freq FT amplitudes in high freq components
Implementation of recombination
• Use butterfly network
• Multiply (time-shift), pad with zeros (duplicate), and sum

Basic 2x2 is repeated

to get NxN inter-connections
 Quadratic speedup
• FFT has N log2 N operations vs. N
2

– quadratic for large N

 2-D Fourier transforms
FT of fuzzy dot
• Magnitude and phase
– both are important input FT

Graphical representation
• phases color coded
• white = 0 amplitude

FT of array of fuzzy dots

input FT
 More 2-D FTs
Convolution in 2-D
•
molecular crystal
–
FT of convolution
–
= product of FTs
Inputs FTs

“molecule”

Array of
molecules
 2D FTs and filtering
Filtered FTs Recovered images
• Spatial filters
• Take FT
• Remove part of FT
• Reconstruct input

Low pass filter

Input 2D-FT

High pass filter

 Magnitude vs. phase
• FT has real and imag parts
• Convert to magnitude and phase
• A sin(2π sx) + B cos(2π sx) = √(A2+B2) cos(2π sx+tan-1 (B/A))

Inputs FTs Mixed FTs Output images

duck Magnitude of duck
phase of cat

cat Magnitude of cat

phase of duck
 Lenses and FTs
Refraction -- Snell’s law
• Lens is position-dependent wedge or prism
• larger refraction angles away from center
Angle
– at lens becomes point at focal plane
• delta function in angle (k-vector) becomes delta function in position x
– like FT
– what applies to delta functions also applies to complicated functions
–

Ray tracings for lens

 Wavelets
• Input functions with discontinuities
– large number of Fourier frequencies needed for edge
– unnecessary computational overhead for rest of signal
• Segment input -- use bandwidth limited sine waves
– 1D analogy -- musical notes vs FT of entire symphony
– can also generalize to more complex basis functions

Truncated sine waves Wavelets

• fixed grid • variable grid
 Sample wavelets
• Popular wavelets -- 4 main families
– Daubechies has fractal character
Top 4 wavelets

Wavelet with fractal character

 Fingerprint ID -- less storage than FT
Wavelet applications
• 0.6 MB each, raw data -- FBI files 200 TB
Data
– smoothing
• do wavelet transform
– threshold and reconstruct
Speech
– synthesis
• choose basis functions to minimize storage/computations
–

Data smoothing
Fingerprint storage/recognition

Speech synthesis