DSP Slides Module8 0

P
ing
i
cess Vetterl Signal Processing
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8: Image Processing
Digi doni a 13
n
0
Pra
2
aolo
Module Overview:
Module 8.2: Ane Transforms
Module 8.3:
Module 8.4:
Module 8.5:
Module 8.1: Introduction to Images and Image Processing
Module 8.6: The JPEG Compression Standard
ing
i
cess Vetterl
ro in
al P art
2D Fourier Analysis l Sign
dM
ta
Digi doni an 13
Image Filters
an 20
lo Pr
Image Compression
Pao
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.1: Image Processing
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
2D signal representations
l
ta
Digi doni a 13
Basic signals and operators ran
0
2
lo P
Pao
Images as multidimensional digital signals
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo P
Pao
Overview:
8.1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo P
Pao
Please meet ...
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.1
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
indices locate a point on a grid pixel al
Sign nd Mar
l
ta
grid is usually regularly spaced
Digi doni a 13
Pran 20
values x[n1 , n2 ] refer to olo pixels appearance
the
Pa
two-dimensional signal x[n1 , n2 ], n1 , n2 Z
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
Sign nd Mar
l
ta
Digi doni a 13
Pran 20
the
Pa
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
Sign nd Mar
l
ta
Digi doni a 13
Pran 20
the
Pa
Digital images
8.1
ing
i
cess Vetterl
ro in
P
t
Sign nd Mar
l
ta
Digi doni a 13
Pran 20
the
Pa
Digital images: grayscale vs color
grayscale images: scalar pixel values
color images: multidimensional pixel values in a color space (RGB, HSV, YUV, etc)
g
8.1
rli
ssin
roce in Vette
we can consider the single components separately:
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
g
8.1
rli
ssin
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
g
8.1
rli
ssin
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
=
+
+
Image processing
ing
i
cess Vetterl
ro in
something still works
al P
t
Sign nd Mar
l
ta
something breaks down
Digi doni a 13
an 20
something is new
lo Pr
Pao
From one to two dimensions...
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
something is new
lo Pr
Pao
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
something is new
lo Pr
Pao
8.1
Image processing
ing
i
cess Vetterl
ro in
linearity, convolution
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
interpolation, sampling lo Pra
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Fourier transform
Digi doni a 13
n
0
2
Pao
What works:
8.1
Image processing
ing
i
cess Vetterl
ro in
Fourier analysis less relevant
al P
t
Sign nd Mar
l
lter design hard, IIRs rare igita
D
ni a
ndo 2013
Pra
linear operators only mildly useful
aolo
P
What breaks down:
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
D
ni a
ndo 2013
Pra
aolo
P
What breaks down:
8.1
Image processing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
D
ni a
ndo 2013
Pra
aolo
P
What breaks down:
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
images are nite-support signals Signa
Mar
l
ta
a d
Digi d their n 13
images are (most often) available inoni entirety causality loses meaning
Pran 20
losignals, designed for a very specic processing system, i.e.
images are very specialized
Pao
new manipulations: ane transforms
the
human brain! Lots of semantics that is extremely hard to deal with
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
a d
Digi d their n 13
Pran 20
Pao
the
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
a d
Digi d their n 13
Pran 20
Pao
the
8.1
Image processing
Whats new:
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
a d
Digi d their n 13
Pran 20
Pao
the
8.1
2D signal processing: the basics
ing
i
cess Vetterl
A two-dimensional discrete-space signal:
ro in
lP
t
gna
Sni2 ], nd 1Mar Z
l1 ,
x[n
n ,n
ta
Digi doni a 13 2
an 20
lo Pr
Pao
8.1
10
2D signals: Cartesian representation
h[n1 , n2 ]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1
8.1
n2
11
2D signals: support representation
ing
i
cess Vetterl
ro in
lP
t
amplitude may be written along Signa
Mar
l
d
ta
explicitly
Digi doni an 13 0
an 20
example:
lo Pr
Pao
n2
just show coordinates of nonzero

samples
[n1 , n2 ] =
1 if n1 = n2 = 0
0 otherwise.
5
5
n1
8.1
12
2D signals: image representation
510
n2
425 ing
i
cess Vetterl
ro 340 in
al P
t
Sign nd Mar
l to
image values are quantized (usually
ta
Digi doni a 13255
8 bits, or 256 levels)
an
20
lo Pr in space
170
the eye does the interpolation
Pao
medium has a certain dynamic range

(paper, screen)
provided the pixel density is high

enough
85
0
0
85
170
255
340
425
510
n1
8.1
13
Why 2D?
8.1
ing
i
cess Vetterl
ro in
images could be unrolled (printers, fax) al P
n
rt
l Sig nd Ma
ta
but what about spatial correlation? ni a
Digi do
13
Pran 20
lo
Pao
14
Why 2D?
8.1
ing
i
cess Vetterl
ro in
images could be unrolled (printers, fax) al P
n
rt
l Sig nd Ma
ta
but what about spatial correlation? ni a
Digi do
13
Pran 20
lo
Pao
14
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
Digi 20doni a0 13 20
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
ing
i
cess Vetterl
ro in
10
al P
t
Sign nd Mar
l
20a
t
10
10
ran 20
olo P
0
Pa
0
0
8.1
205
410
615
820
1025
1230
1435
1640
15
Basic 2D signals: the impulse
[n1 , n2 ] =
1
0
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
if n1 = n2 =al
0
t
0
ia
Digi
otherwise. ndon
013
Pra
2
aolo
P
5
5
8.1
16
Basic 2D signals: the rect
rect
n1 n2
,
2N1 2N2
ing
i
cess Vetterl
ro in
P
t
1 if |n1 | < N1 ignal
Mar
and |n | talN
d
2
Digi < d2oni an 13 0
=
an 20
lo Pr
0 Pao
otherwise;
5
5
8.1
17
Separability
ing
i
cess Vetterl
ro in
lP
t
i 2 a x d 1 ]x2 [n
Sng]n= n1[nMar 2]
x[n1 ,
l
ta
Digi doni a 13
an 20
lo Pr
o
Pa
8.1
18
Separable signals
[n1 , n2 ] = [n1 ][n1 ]ing

s
rli
s
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20 n
n2
o
1
ln1 P,r n2 =
rect o
rect
.
rect
Pa 2N1 2N2
2N1
2N2
8.1
19
Separable signals
[n1 , n2 ] = [n1 ][n1 ]ing

s
rli
s
roce in Vette
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20 n
n2
o
1
ln1 P,r n2 =
rect o
rect
.
rect
Pa 2N1 2N2
2N1
2N2
8.1
19
Nonseparable signal
x[n1 , n2 ] =
1
0
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
if |n1 | + |n2 |ital
<N
Dig doni a 13 0
otherwise
an 20
lo Pr
Pao
5
5
8.1
20
Nonseparable signal
x[n1 , n2 ] = rect
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
ln n
ta
n1 n2
Digi 1 , a2 13 0
,
rect doni
2N1 2N2
2M 2M2
an 1 20
lo Pr
Pao
5
5
8.1
21
2D convolution
ing
i
cess Vetterl
ro in
P
al art
x[n1 , n2 ] h[n1 , n2 ] = l Sign
x[k , k ]h[n k , n k ]
nd M 1 2 1 1 2 2
igitak =i ka= 3
D
on
rand 201
lo P
Pao
1
8.1
22
2D convolution for separable signals
If h[n1 , n2 ] = h1 [n1 ]h2 [n2 ]:
ing
i
cess Vetterl
ro
tin
nal P
Sigh1[nnd M]ar
x[n1 , n2 ] h[n1 , n2 ] =ital
k1
x[k1 , k2 ]h2 [n2 k2 ]
1
Dig k =ni a 13 k =
do
Pran ] 20] x[n , n ]).
lo
1 2
Pao = h1 [n1 (h2[n2
1
8.1
23
2D convolution for separable signals
ing
i
cess Vetterl
ro in
al P
non-separable convolution: M1 M2 operations perart
Sign nd M output sample
tal i a
Digi 2 operations per3
separable convolution: M1 + M ndon
01 output sample!
Pra
2
lo
Pao
If h[n1 , n2 ] is an M1 M2 nite-support signal:
8.1
24
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.1

P
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.2: Image Manipulations
Digi doni a 13
n
0
Pra
2
aolo
Overview:
Ane transforms
Bilinear interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
25
Overview:
Ane transforms
Bilinear interpolation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
25
Ane transforms
mapping R2 R2 that reshapes the coordinate system:
t1
a11 a12 t1 ssing
d1
i
ce dVetterl
= a
t2
al22 rot2
21
2
P
na
rtin
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
t1
t1
d
=A t
t2
2
8.2
26
Ane transforms
mapping R2 R2 that reshapes the coordinate system:
t1
a11 a12 t1 ssing
d1
i
ce dVetterl
= a
t2
al22 rot2
21
2
P
na
rtin
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
t1
t1
d
=A t
t2
2
8.2
26
Translation
A=
1 0
0 1
d=
d1
d2
ing
i
cess Vetterl
ro in
al P
t
=I
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
8.2
27
Translation
A=
1 0
0 1
d=
d1
d2
ing
i
cess Vetterl
ro in
al P
t
=I
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
8.2
27
Scaling
ing
i
cess Vetterl
a1 0
ro in
A=
al P
t
0 a2
Sign nd Mar
l
ta
Digi doni a 13 0
d=0
an
20
o Pr
lpreserved
if a1 = a2 the aspect ratio is
Pao
0
8.2
28
Scaling
ing
i
cess Vetterl
a1 0
ro in
A=
al P
t
0 a2
Sign nd Mar
l
ta
Digi doni a 13 0
d=0
an
20
o Pr
lpreserved
if a1 = a2 the aspect ratio is
Pao
0
8.2
28
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Rotation
A=
d=0
cos
sin
ing
i
cess Vetterl
ro in
al P
t
sin
Sign nd Mar
l
ta
0
cos igi
D
ni a
ndo 2013
Pra
aolo
P
0
8.2
29
Flips
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 0
0 1
1 P0
0 1
d=0
0
8.2
30
Flips
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 0
0 1
1 P0
0 1
d=0
0
8.2
30
Shear
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 s
0 1
1 P0
s 1
d=0
0
8.2
31
Shear
Horizontal:
A=
d=0
Vertical:
A=
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13 0
an 20
lo Pr
ao
1 s
0 1
1 P0
s 1
d=0
0
8.2
31
Ane transforms in discrete-space
ing
i
cess Vetterl
ro in
al P
t
t1
S nn n d r
= lA ig 1 d MaR2 = Z2
ta
tg
n
Di2 i don2i a 13
an 20
lo Pr
ao
8.2
32
Solution for images
8.2
apply the inverse transform:
i
t1
m +sdng
li
= A1 roc1es 1 ; etter
t2
l P m2 + dn V
i2
na
rt
l Sig to nd Ma
interpolate from the originaligitapoint a the mid-point
grid
D
ni
ndo 2013
(t1 , t2 ) =lo 1 + 1 , 2 + 2 ),
( Pra
1,2 Z,
ao
P
0 1,2 < 1
33
Solution for images
8.2
apply the inverse transform:
i
t1
m +sdng
li
= A1 roc1es 1 ; etter
t2
l P m2 + dn V
i2
na
rt
l Sig to nd Ma
interpolate from the originaligitapoint a the mid-point
grid
D
ni
ndo 2013
(t1 , t2 ) =lo 1 + 1 , 2 + 2 ),
( Pra
1,2 Z,
ao
P
0 1,2 < 1
33
Bilinear Interpolation
x[1 + 1, 2 + 1]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an
20
lo Pr x[ ,
]
ao
2 + 1
1
8.2
1 + 1
34
x[1 + 1, 2 + 1]
ing
i
cess Vetterl
(t , t )ro
al P
tin
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
2 + 1
Pa
x[1 , 2 ]
1
8.2
1 + 1
34
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi donia 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi donia 13
an 20
lo Pr
ao
2 + 1
1
8.2
1 + 1
34
ing
i
cess Vetterl
ro in
al P
t
y [m1 , m2 ] = (1 1 )(1 ig2 )x[1 , 2 ] ar1 (1 2 )x[1 + 1, 2 ]
S n nd M +
l
igita on a
+ (1 D 1 )2 x[1 , i 2 + 1] + 1 2 x[1 + 1, 2 + 1]
d
13
Pran 20
lo
Pao
If we use a rst-order interpolator:
8.2
35
Shearing
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.2
36
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.2

P
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.3: Frequency Analysis
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.3
ing
i
cess Vetterl
ro in
DFT
al P
t
Sign nd Mar
l
ta
Magnitude and phase
Digi doni a 13
an 20
lo Pr
Pao
37
Overview:
8.3
ing
i
cess Vetterl
ro in
DFT
al P
t
Sign nd Mar
l
ta
Magnitude and phase
Digi doni a 13
an 20
lo Pr
Pao
37
2D DFT
g j rni k
l
c
ro in Vette
n =0 n =0 l P
na
rt
l Sig nd Ma
ta
Digi doni a 13
an N 20
1
lo Pr N 1 1 X [k , k ]e j n k e j n k
x[n1Pn2o=
, a]
1 2
N1 N2
N1 1 N2 1
X [k1 , k2 ] =
x[n1 , n2 ]e ssin1 1 e
e N1
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
2D DFT
g j rni k
l
c
ro in Vette
n =0 n =0 l P
na
rt
l Sig nd Ma
ta
Digi doni a 13
an N 20
1
lo Pr N 1 1 X [k , k ]e j n k e j n k
x[n1Pn2o=
, a]
1 2
N1 N2
N1 1 N2 1
X [k1 , k2 ] =
x[n1 , n2 ]e ssin1 1 e
e N1
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
2D-DFT Basis Vectors
ing
i
cess Vetterl
ro in
al P n k t n k
j
j
wk ,k [n1Sign= e Mar
, n2 ]
e
ital ni and
Dig do
13
Pran 20
olo
There are N1 N2 orthogonal basis vectors for an N1 N2 image:
2
N1
1 1
2
N2
2 2
P
for n1 , k1 = 0, 1, . . . , N1 1 and n2 , k2 = 0, 1, . . . , N2 1
8.3
39
2D-DFT basis vectors for k1 = 1, k2 = 0 (real part)

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
40

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
41

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
42

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
43

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
44

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
45

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
46

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
47

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
48

255
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
0
0
8.3
255
49
2D DFT
2D-DFT basis functions are separable, and so is the 2D-DFT:
g
i
s in
cejs n Vetterln k
o
rn2 ]e in k e j
X [k1 , k2 ] =
x[nP,
al 1
t
Sign nd Mar
n al n =0
=0
t
Digi doni a 13
an 20
1D-DFT along n2 (the columns)
lo Pr
Pao
N1 1 N2 1
8.3
2
N1
1 1
2
N2
2 2
1D-DFT along n1 (the rows)
50
2D DFT
ing e i
cess n Vettjrl n k
o
X [k1 , k2 ] =
x[nPr 2 ]e
l , n j k e
gna 1 Martin
n =0 l Si=0
n
igita oni and 3
D
d
1
Pran 20
lo
Pao
N1 1
8.3
N2 1
2
N2
2 2
2
N1
1 1
50
2D DFT
sin
li
o ]e s k t er
rn2cej inn Vete j
x[nP,
X [k1 , k2 ] =
al 1
t
n =0
n =0
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
N2 1
N1 1
8.3
2
N2
2 2
2
n k
N1 1 1
50
2D DFT
g r
ssin k ttejli
ocej n
X [k1 , k2 ] =
x[nP,rn2 ]e in Ve e
al 1 art
n =0
n =0
Sign
ital ni and M
ig
D
do
13
Pran 20
lo
Pao
N1 1
8.3
N2 1
2
N2
2 2
2
n k
N1 1 1
50
2D DFT in matrix form
nite-support 2D signal can be written as a matrix x
N1 N2 image is an N2 N1 matrix (n1 spans the columns, n2 spans the rows)

g
8.3
essi
terli
et
recall also the N N DFT matrix (ModuleProc
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51

g
8.3
essi
terli
et
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51

g
8.3
essi
terli
et
al 4.2): artin V
n
l Sig 1 nd M 1
1 igita
1
1
i a2 3 3 . . .
N1
1 D W 1ndonW 01 W
. . . WN
a
N2
lo Pr 2
2(N1)
6
W4
WN
. . . WN
WN = ao
1 WN
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
n
013
Pra
2
lo
V ao N N
P R
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
3 X = V WN
n
201
ra
lo P
V ao N N
X RN N
P R
n1 =0
n2 =0
8.3
52
N1 1
X [k1 , k2 ] =
N2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
a
Digi
V = WN x doni
3 X = V WN
n
201
ra
lo P
V ao N N
X RN N
P R
n1 =0
n2 =0
X = WN2 x WN1
8.3
52
How does a 2D-DFT look like?
8.3
ing
i
cess Vetterl
ro in
P range
t
problem: the range is too big for theignal
S grayscaleMar of paper or screen
l
ita n and |X
try to normalize: |X [n1 , nDig |X [no,ni2 ]|/ max3 [n1 , n2 ]|
2 ]| =
d1
1
Pran 20
lo
but it doesnt work... ao
P
try to show the magnitude as an image
53
DFT coecients sorted by magnitude
107
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
105
103
101
0
8.3
5122
54
Dealing with HDR images
sing
li
lP
tin V
remove agrant outliers (e.g. X [0, 0]igna
1
S = nd x[na,rn2])
tal 1/3 a M
Digiy =doni after normalization (x 1)
use a nonlinear mapping: e.g.
x
13
Pran 20
lo
Pao
s
if the image is high dynamic range we need to compress the levels r
ette
roce
8.3
55
Dealing with HDR images
sing
li
lP
tin V
remove agrant outliers (e.g. X [0, 0]igna
1
S = nd x[na,rn2])
tal 1/3 a M
Digiy =doni after normalization (x 1)
use a nonlinear mapping: e.g.
x
13
Pran 20
lo
Pao
s
if the image is high dynamic range we need to compress the levels r
ette
roce
8.3
55
How does a 2D-DFT look like?

510
510
425
425
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
255
ta
Digi doni a 13
an 20 170
lo Pr
ao
340
340
255
170
85
85
0
0
8.3
85
170
255
340
425
510
85
170
255
340
425
510
56
DFT magnitude doesnt carry much information
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.3
57
DFT phase, on the other hand...
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.3
58
Image frequency analysis
8.3
most of the information is contained in images edgesng

ssi
rli
roce in Vette
P
edges are points of abrupt change inignal values t
S signals Mar
ital and
edges are a space-domain ig
D feature oninot captured by DFTs magnitude
13
ra d
Pfor n 20edges
lo
phase alignment is important
reproducing
Pao
59
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.3

P
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.4: Filtering
Digi doni a 13
n
0
Pra
2
aolo
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Filters for image processing
Classication
Examples
8.4
60
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Classication
Examples
8.4
60
Overview:
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Classication
Examples
8.4
60
Analogies with 1D lters
space invariance
impulse response
frequency response
stability
8.4
linearity
2D CCDE
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
61
The problem with LSI operators
interesting images contain lots of semantics: dierent information in dierent areas
n
space-invariant lters process everything in the same iway
ess
g
rli
c
ro in Vette
P
t
but we should process things dierently al
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
n
ess
g
rli
c
ro in Vette
P
t
Sign nd Mar
l
ta
edges
Digi doni a 13
an 20
gradients
lo Pr
Pao
textures
...
8.4
62
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
highpass, lowpass, ...
Digi doni a 13
lowpass image smoothing an
0
2
lo Pr
highpass enhancement, edge detection
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo Pr
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo Pr
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo Pr
Pao
IIR, FIR
63
Filter types
8.4
ing
i
cess Vetterl
ro in
causal or noncausal
al P
t
Sign nd Mar
l
ta
Digi doni a 13
0
2
lo Pr
Pao
IIR, FIR
63
The problems with 2D IIRs
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
ita algebra doesnt hold in multiple dimensions!
stability: the fundamental Dig
theorem of ni a
do
13
Pran 20
lo
computability
Pao
nonlinear phase (edges!)
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
theorem of ni a
do
13
Pran 20
lo
computability
Pao
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
theorem of ni a
do
13
Pran 20
lo
computability
Pao
64
8.4
ing
i
cess Vetterl
ro in
al P
t
border eects
Sign nd Mar
l
theorem of ni a
do
13
Pran 20
lo
computability
Pao
64
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
2
y[0, 1]
y[0, 0]
y[1, 0]
y[1, 0]
y[0, 1]
1
2
8.4
2
65
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
ing
i
cess Vetterl
ro in
al P
t
1
Sign nd Mar
l
ta
Digi doni a 13
an 20
0
lo Pr
ao
2
8.4
2
65
Practical FIR lters
8.4
generally zero centered (causality not an issue)

ing
odd number of taps in both directions
cess
rli
ro in Vette
al P
t
per-sample complexity:
Sign nd Mar
lresponses
ta
M1 M2 for nonseparable impulse
Digi doni a 13
an 0
M1 + M2 for separable impulse responses 2
lo Pr
Pao
obviously always stable
66
Practical FIR lters
8.4

ing
cess
rli
ro in Vette
al P
t
Sign nd Mar
lresponses
ta
Digi doni a 13
an 0
lo Pr
Pao
66
Practical FIR lters
8.4

ing
cess
rli
ro in Vette
al P
t
Sign nd Mar
lresponses
ta
Digi doni a 13
an 0
lo Pr
Pao
66
Practical FIR lters
8.4

ing
cess
rli
ro in Vette
al P
t
Sign nd Mar
lresponses
ta
Digi doni a 13
an 0
lo Pr
Pao
66
Practical FIR lters
8.4

ing
cess
rli
ro in Vette
al P
t
Sign nd Mar
lresponses
ta
Digi doni a 13
an 0
lo Pr
Pao
66
Moving Average
sing n i
cesx[n1 Vek1t,er2l k2]
ro in t
k =N k =N
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
n n
1
y [n1 , n2 ] =
1
(2N + 1)2
Pah[n1, n2 ] =
8.4
(2N +
1)2
rect
2N 2N
67
Moving Average
sing n i
cesx[n1 Vek1t,er2l k2]
ro in t
k =N k =N
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
o
1
n n
y [n1 , n2 ] =
1
(2N + 1)2
Pah[n1, n2 ] =
8.4
(2N +
1)2
rect
2N 2N
67
Moving Average
sing
ter
o es
Pr1 c 1 t1n Vet
l
na1
ri
h[n1 , Sig = d 1 a 1
l n2 ] 9 M 1
ta
n
Digi
oni a 1 31 1
d
an
lo Pr
ao
8.4
li
01
68
Moving Average
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
11 11 MA
8.4
51 51 MA
69
Gaussian Blur
ing
i
cess Vetterl
1
r,o |nn, n | < N
h[n1 , n2 ] =
e al P
ti 1 2
2gn
Si2 nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Pao
with N 3
n2 +n2
1 22
2
8.4
70
Gaussian Blur
h[n1 , n2 ]
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
n1
8.4
n2
71
Gaussian Blur
12
8
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
0
l
ta
Digi doni a 13
4
an 20
lo Pr
Pao
8
4
12
12 8
8.4
4
0
4
= 5, N = 14
12
72
Gaussian Blur
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
= 1.8, 11 11 blur
8.4
= 8.7, 51 51 blur
73
Sobel lter
approximation of the rst derivative in the horizontal direction:
1 0 1
i g
i
so [n1 , n2 ] = 2 ess2n
c 0 1 Vetterl
ro 0 in
1
al P
n
rt
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
separability and structure:P
8.4

1
so [n1 , n2 ] = 2 1 0 1
1
74
Sobel lter
approximation of the rst derivative in the horizontal direction:
1 0 1
i g
i
so [n1 , n2 ] = 2 ess2n
c 0 1 Vetterl
ro 0 in
1
al P
n
rt
l Sig nd Ma
ta
Digi doni a 13
an 20
lo Pr
ao
separability and structure:P
8.4

1
so [n1 , n2 ] = 2 1 0 1
1
74
Sobel lter
approximation of the rst derivative in the vertical direction:

ing
i
cess etterl
ro in V
lP
a1 art 1
Sign nd M2 0
s tal n2 i
Digvi [n1 ,do]n=a 013 0 1
1
an
20 2
lo
Pao
8.4
Pr
75
Sobel lter
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
horizontal Sobel lter

8.4
vertical Sobel lter

76
Sobel operator
approximation for the square magnitude of the gradient: sing
rli
s
roce in Vette
|x[n1 , n2 ]|2 = |so [n1 , n2 ] gnal1 P 2 ]|2art v [n1 , n2 ] x[n1 , n2 ]|2
x[n , n
+ |s
M
l Si
d
ta
Digi doni an 13
(operator because its nonlinear)
an 20
lo Pr
Pao
8.4
77
Gradient approximation for edge detection
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Sobel operator
8.4
thresholeded Sobel operator

78
Laplacian operator
ing
i
cess Vetterl
ro in
al P f a t
S,ign =n 2M+ r 2 f
f (t1 t2 ) d 2
tal
2
Digi doni a t13 t2
n
01
ra
2
olo P
Laplacian of a function in continuous-space:
Pa
8.4
79
Laplacian operator
approximating the Laplacian; start with a Taylor expansion
f (t + ) =
f (n) (t) n
g
n! ssin
e
rli
roc in Vette
al P
t
Sign nd Mar
l
ita (t
and compute the expansion in Di+ ) andni a ): 3
(t g
do
1
Pran 20
lo
Pao(t + ) = f (t) + f (t) + 1 f (t) 2
f
n=0
2
1
f (t ) = f (t) f (t) + f (t) 2
2
8.4
80
Laplacian operator
by rearranging terms:
dM
tal
Digi doni an 13
an 20
which, on the discrete grid, is lo Pr h[n] = 1 2
the FIR
Pao
8.4
ssin
rli
t
1
oce + f V + te
f (t) = 2 (f (t a) r2f (t) tin (t e ))
l P
Sign
ar
1
81
Laplacian
summing the horizontal and vertical components:

ing
i
cess Vetterl
ro in
l
aP a t
Sign 0 M1 r 0
l , n2 ] = n1 4 1
d
ta
Digih[n1doni a 0 131 0
an
20
lo
Pao
8.4
Pr
82
Laplacian
ing
i
cess Vetterl
ro
al P 1 i
g = 1 8 t1n
Si2 ]n 1 Mar 1
h[n1 , n
d
tal
Digi doni an1 131 1
an 20
lo Pr
o
If we use the diagonals too:
Pa
8.4
83
Laplacian for Edge Detection
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
Laplacian operator
8.4
thresholeded Laplacian operator

84
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.4

P
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Module 8.5: Image Compression
Digi doni a 13
n
0
Pra
2
aolo
Overview:
8.5
ing
i
cess Vetterl
ro in
Redundancy in natural images
al P
t
Sign nd Mar
l
Image coding ingredients Digita
ni a
ndo 2013
Pra
aolo
P
85
Overview:
8.5
ing
i
cess Vetterl
ro in
Redundancy in natural images
al P
t
Sign nd Mar
l
Image coding ingredients Digita
ni a
ndo 2013
Pra
aolo
P
85
A thought experiment
8.5
ing
i
cess Vetterl
ro in
each image is 524,288 bits
al P
t
Sign nd Mar
l
ta
Digi 2doni a 10157,826
total number of possible images: n 524,288 2013
Pra
aolo
P
number of atoms in the universe: 1082
consider all possible 256 256, 8bpp images
86
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Pra
aolo
P
86
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Pra
aolo
P
86
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Pra
aolo
P
86
How many bits per image?

Another thought experiment
take all images in the world and list them in an encyclopedia of images
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
on the Internet: M = 50 billiont
Digi doni a 13
an
20
raw encoding: 524,288 bits per image
lo Pr
Pao
to indicate an image, simply give its number
8.5
enumeration-based encoding: log2 M 36 bits per image

(of course, side information is HUGE)
87

ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
Digi doni a 13
an
20
lo Pr
Pao
8.5

87

ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
Digi doni a 13
an
20
lo Pr
Pao
8.5

87

ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
Digi doni a 13
an
20
lo Pr
Pao
8.5

87

ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
Digi doni a 13
an
20
lo Pr
Pao
8.5

87

ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
a
Digi doni a 13
an
20
lo Pr
Pao
8.5

87
Compression
ing
i
cess Vetterl
ro in
nal P M rt
allocate bits for things that matter Sig edges) a
(e.g.
ital ni d
Digto ndooutan 1matters
use psychovisual experiments and
what 3
0
Pr
2
lo
Pao
some information is discarded: lossy compression
Another approach:
8.5
exploit physical redundancy
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
(e.g.
ital ni d
what 3
0
Pr
2
lo
Pao
Another approach:
8.5
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
(e.g.
ital ni d
what 3
0
Pr
2
lo
Pao
Another approach:
8.5
88
Compression
ing
i
cess Vetterl
ro in
nal P M rt
(e.g.
ital ni d
what 3
0
Pr
2
lo
Pao
Another approach:
8.5
88
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
using a suitable transform (i.e., a change of basis) rt
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
compressing at block level
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
89
Key ingredients
8.5
ing
i
cess Vetterl
ro in
al P
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
89
Compressing at pixel level
8.5
ing
i
cess Vetterl
reduce number bits per pixel
ro in
al P
t
Sign nd Mar
l
equivalent to coarser quantization
ta
Digi doni a 13
in the limit, 1bpp
an 20
lo Pr
Pao
90
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
equivalent to coarser quantization
ta
Digi doni a 13
in the limit, 1bpp
an 20
lo Pr
Pao
90
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
equivalent to coarser quantization oni a
Digi d
13
Pran 20
lo
in the limit, 1bpp
Pao
90
Compressing at block level
8.5
ing
i
cess Vetterl
ro in
al P art
code the average value with 8 bits l Sign
ita n d M
Diggivesdlessi an 13
o
3 3 blocks at 8 bits per block
Pran 20
than 1bpp
lo
Pao
divide the image in blocks
91
8.5
ing
i
cess Vetterl
ro in
al P art
code the average value with 8 bits l Sign
ita n d M
Diggivesdlessi an 13
o
3 3 blocks at 8 bits per block
Pran 20
than 1bpp
lo
Pao
91
8.5
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
t
code the average value with 8 ibitsa
D gi doni a 13
an
3 3 blocks at 8 bits per block gives less 20
lo Pr
than 1bpp
Pao
91
8.5
ing
i
cess Vetterl
ro in
exploit the local spatial correlation
al P
t
Sign nd Mar
l
ta
compress remote regions independentlyi a
Digi don
13
Pran 20
lo
Pao
92
8.5
ing
i
cess Vetterl
ro in
exploit the local spatial correlation
al P
t
Sign nd Mar
l
ta
compress remote regions independentlyi a
Digi don
13
Pran 20
lo
Pao
92
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
storing the signal requires NR bitsSigna
Mar
l
ta
i d
Digi d like an 13
now you take the DFT and it lookson
an 20
this
lo Pr
Pao
take a DT signal, assume R bits per

sample
in theory, we can just code the two

nonzero DFT coecients!
10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
i d
Digi d like an 13
an 20
this
lo Pr
Pao

sample

10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
i d
Digi d like an 13
an 20
this
lo Pr
Pao

sample

10
15
20
25
30
93
Transform coding
A simple example:
8.5
ing
i
cess Vetterl
ro in
lP
t
Mar
l
ta
i d
Digi d like an 13
an 20
this
lo Pr
Pao

sample

10
15
20
25
30
93
Transform coding
i
sing
cesin a Vetterl
captures the important features of an image ro
block
few coecients
al P
tin
Sign nd Mar
l
is ecient to compute
ta
Digi doni a 13
an 20
lo Pr
answer: the Discrete Pao Transform
Cosine
Ideally, we would like a transform that:
8.5
94
Transform coding
i
sing
cesin a Vetterl
block
few coecients
al P
tin
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Cosine
8.5
94
Transform coding
i
sing
cesin a Vetterl
block
few coecients
al P
tin
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
Cosine
8.5
94
2D-DCT
ing
i
cess Vetterl
ro in
N1 N1
al P
rt
Sign nd 1 +a1 k1 cos
C [k1 , k2 ] =
x[n1 , n2l] cos
nM
ita ni N
2
N
a
n =0 n =0Dig
do 2013
an
lo Pr
Pao
1
8.5
n2 +
1
2
k2
95
DCT basis vectors for an 8 8 image
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.5
96
Smart quantization
8.5
ing
i
cess Vetterl
ro in
deadzone
al P
t
Sign nd Mar
l
variable step (ne to coarse) gita
Di
ni a
ndo 2013
Pra
aolo
P
97
Smart quantization
8.5
ing
i
cess Vetterl
ro in
deadzone
al P
t
Sign nd Mar
l
variable step (ne to coarse) gita
Di
ni a
ndo 2013
Pra
aolo
P
97
Quantization
x [n]
ing
i
cess Ve1terl
ro in t 00
Standard quantization:
al P
t
Sign nd Mar
l
ita ni a
x = oor(x) + 0.5 Dig
do
013 1 10
Pran 22
lo
Pao
01
x[n]
1
1
11
2
8.5
98
Quantization
x [n]
Deadzone quantization:
ing
i
cess Ve1terl
ro in t
al P
t
Sign nd Mar
l
ta
00
Digi doni a 13
0
an 22
1
lo Pr
ao
x = round(x)
01
x[n]
1
10
8.5
98
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
associate short symbols to frequentSvalues
vice-versa
l i
igita oni and 3
D
d
01
Pr n
if it sounds familiar its becauseait is... 2
lo
Pao
minimize the eort to encode a certain amount cessi

of informationrli
tte
99
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
vice-versa
l i
igita oni and 3
D
d
01
Pr n
lo
Pao

of informationrli
tte
99
Entropy coding
8.5
ng
Ve
l Pro
gna andMartin
vice-versa
l i
igita oni and 3
D
d
01
Pr n
lo
Pao

of informationrli
tte
99
Entropy coding
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.5
100
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.5

P
ing
i
ro in Digital
al P
t
Sign nd Mar
l
ta
Digi doni a Module 8.6: The JPEG Compression Algorithm
13
Pran 20
aolo
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
Sign nd Ma
tal
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
split image into 8 8 non-overlapping blocks
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
compute the DCT of each block
Sign nd Mar
l
ta
smart quantization
Digi doni a 13
an 20
lo Pr
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ita to a
quantize DCT coecients Dig
accordingonipsycovisually-tuned tables
d
13
Pran 20
lo
entropy coding
Pao
101
Key ingredients
8.6
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ita to a
quantize DCT coecients Dig
accordingonipsycovisually-tuned tables
d
13
Pran coding 20
lo
run-length encoding andoHuman
Pa
101
DCT coecients of image blocks (detail)
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
102
DCT coecients of image blocks (detail)
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
102
Smart quantization
8.6
most coecients are negligible captured by the deadzone

ing
rli
ss
roce in Vette
some coecients are more important thanl others t
a P
Sign nd Mar
al
nd out the critical coecientstby experimentation
Digi doni a 13
an 20
allocate more bits (or, equivalently, ner quantization levels) to the most important
lo Pr
Pao
coecients
103
Smart quantization
8.6

ing
rli
ss
roce in Vette
a P
Sign nd Mar
al
Digi doni a 13
an 20
lo Pr
Pao
coecients
103
Smart quantization
8.6

ing
rli
ss
roce in Vette
a P
Sign nd Mar
al
Digi doni a 13
an 20
lo Pr
Pao
coecients
103
Smart quantization
8.6

ing
rli
ss
roce in Vette
a P
Sign nd Mar
al
Digi doni a 13
an 20
lo Pr
Pao
coecients
103
Psychovisually-tuned quantization table

c [k1 , k2 ] = round(c[k1 , k2 ]/Q[k1 , k2 ])
g
16 11 10 16 24 essin 51 terli
c 40 Vet 61
12 12 14 19 l Pro 58n 60 55
a 26 ti
14 13 l 16gn
Si 24 nd40 ar57 69 56
M
gita
14 i 17 22 ni a 51 87 80 62
D
do 29 2013 109 103 77
Q=
ran
18 P22 37 68
lo 35 55 56 81 104 113 92
o
a24
64
P
49 64 78 87 103 121 120 101

72 92 95 98 112 100 103 99
8.6
104
Advantages of nonuniform bit allocation
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
uniform
8.6
tuned
105
Advantages of nonuniform bit allocation (detail)
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
uniform
8.6
tuned
106
Ecient coding
8.6
sing terli
et
al P artin V
Si n
use zigzag scan to maximize orderingg
ital ni and M
Dig
quantization will create long seriesdo zeros2013
an of
lo Pr
Pao
most coecients are small, decreasing with indexes
roc
107
Ecient coding
8.6
sing terli
et
al P artin V
Si n
ital ni and M
Dig
an of
lo Pr
Pao
roc
107
Ecient coding
8.6
sing terli
et
al P artin V
Si n
ital ni and M
Dig
an of
lo Pr
Pao
roc
107
Zigzag scan
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
108
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 essi0 g0 erli
0 n
t
0
0 0 l 0 ro 0 i0 Vet
P c n
a 0 0 a0rt 0 0
13 1gn
0
0
ital Si 0 nd M
ig0
0 ni0a 0 0 0 0 0
D
0 ando 0 2013 0 0 0
r 0 0 0
lo P
0 0 0 0 0 0 0
Pao 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
109
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 essi0 g0 erli
0 n
t
0
0 0 l 0 ro 0 i0 Vet
P c n
a 0 0 a0rt 0 0
13 1gn
0
0
ital Si 0 nd M
ig0
0 ni0a 0 0 0 0 0
D
0 ando 0 2013 0 0 0
r 0 0 0
lo P
0 0 0 0 0 0 0
Pao 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
109
Runlength encoding
Each nonzero value is encoded as the triple
ing
i
cess Vetterl
ro in
nal P M rt
r is the runlength i.e. the numberl ofig
S zerosnbeforeathe current value
d
ta
Digi bits oni a to1encode the value
d needed20 3
s is the size i.e. the number of an
Pr
aolo
c is the actual value P
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
d
ta
d needed20 3
Pr
aolo
[(r , s), c]
8.6
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
d
ta
d needed20 3
Pr
aolo
[(r , s), c]
8.6
110
Runlength encoding
ing
i
cess Vetterl
ro in
nal P M rt
d
ta
d needed20 3
Pr
aolo
[(r , s), c]
8.6
110
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 0 0 g0
in i
0
0 0 0 ocess 0 0 terl
r 0 0 in Vet
P
13 1 0 l 0 0 0 t 0 0
gna Mar
i
0tal S0 0 0d 0 0 0 0
n
igi
D
0 doni0a 0 03 0 0 0
0
ran 0 201 0 0 0
0 0 0
lo P0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 0 0 g0
in i
0
0 0 0 ocess 0 0 terl
r 0 0 in Vet
P
13 1 0 l 0 0 0 t 0 0
gna Mar
i
0tal S0 0 0d 0 0 0 0
n
igi
D
0 doni0a 0 03 0 0 0
0
ran 0 201 0 0 0
0 0 0
lo P0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
The runlength-size pairs
8.6
ing
i
cess Vetterl
ro in
al P
t
in theory, 8 bits per pair
Sign nd Mar
l
ita than others!
some pairs are much moreDig
common oni a
and 13
Prbeing smart20
lo
a lot of space can be saved by
Pao
by design, (r , s) A with |A| = 256
112
8.6
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ita than others!
common oni a
and 13
Prbeing smart20
lo
Pao
112
8.6
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ita than others!
common oni a
and 13
Prbeing smart20
lo
Pao
112
8.6
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ita than others!
common oni a
and 13
Prbeing smart20
lo
Pao
112
Variable-length encoding
great idea: shorter binary sequences for common symbols
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
8.6
113
however: if symbols have dierent lengths, we mustg

sin know how to parse them!
li
8.6
tter
oces
gn
ar
al Si i a d M
itlettersnandnwords (wasteful)
in Morse code, pauses separate
Dig do
13
Pran 20
can we do away with separators?
lo
Pao
e
in English, spaces separate words extral symbol (wasteful)
a Pr
tin V
114

li
8.6
tter
oces
gn
ar
al Si i a d M
Dig do
13
Pran 20
lo
Pao
e
a Pr
tin V
114

li
8.6
tter
oces
gn
ar
al Si i a d M
Dig do
13
Pran 20
lo
Pao
e
a Pr
tin V
114
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
can parse a bitstream sequentiallyl with
look-ahead
ta
Digi doni a 13
extremely easy to understand graphically... 20
an
lo Pr
Pao
li
r
no valid sequence can be the beginning of anothersvalid sequence
ette
roce
115
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
look-ahead
ta
Digi doni a 13
an
lo Pr
Pao
li
r
ette
roce
115
Prex-free codes
8.6
sing
al P
tin V
Signnond Mar
look-ahead
ta
Digi doni a 13
an
lo Pr
Pao
li
r
ette
roce
115
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
A
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AAB
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAAB
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABA
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABAD
8.6
116
Prex-free code
0
0
C
sing terli
s
0
oce n et
al Pr arti1 V D
1
Sign
ital ni and M
Dig do 1 13B
an 20
lo Pr
ao
001100110101100
AABAABADC
8.6
116
Entropy coding
ing
i
cess Vetterl
assign short sequences to more frequent symbols
ro in
al P
t
Sign nd Mara set of symbol probabilities
the Human algorithm builds the loptimal code for
ta
Digi doni a 13
in JPEG, you can use a general-purpose 20
an Human code or build your own
lo Pr
(but then you pay a side-information price)
Pao
goal: minimize message length
8.6
117
Entropy coding
ing
i
cess Vetterl
ro in
al P
t
ta
Digi doni a 13
lo Pr
Pao
8.6
117
Entropy coding
ing
i
cess Vetterl
ro in
al P
t
ta
Digi doni a 13
lo Pr
Pao
8.6
117
Example
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l 0.38
igita
Dp(A) =doni a 13 p(B) = 0.32
20
Pran
p(D) = 0.2
lop(C ) = 0.1
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
Example
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l 0.38
igita
Dp(A) =doni a 13 p(B) = 0.32
20
Pran
p(D) = 0.2
lop(C ) = 0.1
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
Building the Human code
C
0.10 sing
p(B) = 0.32
p(C ) = 0.1
rli
s
roce in Vette
P
t
0.30gnal
Si
Mar
l
d
ta
Digi doni an 0.20 D
1
13
Pran 20
aolo
p(A) = 0.38
8.6
p(D) = 0.2
119
Building the Human code
ng
i
e i e
c1 ss VDtterl
ro in
al P
0.62
t
Sign nd Mar
l
ta
Digi doni a 1B
1 0.32 3
ran 20
P
aolo
0
0.30
p(A) = 0.38
8.6
p(B) = 0.32
p(C + D) = 0.3
120
Human Coding
0.38 A
ing
i
cess Vetterl
ro in0
al P art
1.00
C
1 l Sign
n
ita ni a0 d M
Dig do
13
D
1
Pran 0.62 20
lo
Pao
0
p(A) = 0.38
8.6
p(B + C + D) = 0.62
121
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8.6

P
ing
i
cess Vetterl
ro in
al P
t
Sign nd Mar
l
ta
Digi doni a 13
an 20
lo Pr
ao
END OF MODULE 8
P

DSP Slides Module8 0

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DSP Slides Module8 0

Transféré par

Droits d'auteur :

Formats disponibles

P

Module 8.2: Ane Transforms

Module 8.1: Introduction to Images and Image Processing

Module 8.6: The JPEG Compression Standard

Please meet ...

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

Digital images: grayscale vs color

grayscale images: scalar pixel values

Digital images: grayscale vs color

grayscale images: scalar pixel values

Digital images: grayscale vs color

grayscale images: scalar pixel values

From one to two dimensions...

From one to two dimensions...

From one to two dimensions...

What breaks down:

What breaks down:

What breaks down:

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

2D signal processing: the basics

2D signals: Cartesian representation

2D signals: support representation

just show coordinates of nonzero

2D signals: image representation

medium has a certain dynamic range

provided the pixel density is high

Basic 2D signals: the impulse

Basic 2D signals: the rect

[n1 , n2 ] = [n1 ][n1 ]ing

[n1 , n2 ] = [n1 ][n1 ]ing

2D convolution for separable signals

If h[n1 , n2 ] = h1 [n1 ]h2 [n2 ]:

2D convolution for separable signals

If h[n1 , n2 ] is an M1 M2 nite-support signal:

END OF MODULE 8.1

Ane transforms in discrete-space

Solution for images

apply the inverse transform:

Solution for images

apply the inverse transform:

If we use a rst-order interpolator:

END OF MODULE 8.2

2D-DFT Basis Vectors

There are N1 N2 orthogonal basis vectors for an N1 N2 image:

2D-DFT basis vectors for k1 = 1, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 0, k2 = 1 (real part)

2D-DFT basis vectors for k1 = 2, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 3, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 0, k2 = 3 (real part)

2D-DFT basis vectors for k1 = 30, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 1, k2 = 1 (real part)

2D-DFT basis vectors for k1 = 2, k2 = 7 (real part)

2D-DFT basis vectors for k1 = 5, k2 = 250 (real part)

2D-DFT basis vectors for k1 = 3, k2 = 230 (real part)

2D-DFT basis functions are separable, and so is the 2D-DFT:

1D-DFT along n1 (the rows)

2D-DFT basis functions are separable, and so is the 2D-DFT: