Vous êtes sur la page 1sur 81

EECE/CS 253 Image Processing

Richard Alan Peters II


Department of Electrical Engineering and
Computer Science
Fall Semester 2011
Lecture Notes: Introduction and Overview
This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or
send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.
. 1 % { . { . . . . .
Introduction and Overview
This presentation is an
overview of some of the
ideas and techniques to be
covered during the course.
25 August 2011 1999-2011 by Richard Alan Peters II 2
1. Image formation
2. Point processing and equalization
3. Color correction
4. The Fourier transform
5. Convolution
6. Image sampling, warping, and stitching
7. Spatial filtering
8. Noise reduction
9. Mathematical morphology
10. High dynamic range imaging
11. Image compression
Topics
25 August 2011 1999-2011 by Richard Alan Peters II 3
Wallace and Gromit
Wallace
Gromit
likes cheese
reads Electronics for Dogs
http://www.aardman.com/wallaceandgromit/index.shtml
Wallace and Gromit will be subjects of some of the imagery in this introduction.
25 August 2011 1999-2011 by Richard Alan Peters II 4
Image Formation
25 August 2011 1999-2011 by Richard Alan Peters II 5
Image Formation
25 August 2011 1999-2011 by Richard Alan Peters II 6
Image Formation
projection
through lens
image of object
25 August 2011 1999-2011 by Richard Alan Peters II 7
Image Formation
projection onto
discrete sensor
array.
digital camera
25 August 2011 1999-2011 by Richard Alan Peters II 8
Image Formation
sensors register
average color.
sampled image
25 August 2011 1999-2011 by Richard Alan Peters II 9
Image Formation
continuous colors,
discrete locations.
discrete real-
valued image
25 August 2011 1999-2011 by Richard Alan Peters II 10
Digital Image Formation: Quantization
continuous color input
d
i
s
c
r
e
t
e

c
o
l
o
r

o
u
t
p
u
t

continuous colors
mapped to a finite,
discrete set of colors.
25 August 2011 1999-2011 by Richard Alan Peters II 11
Sampling and Quantization
pixel grid
sampled real image quantized sampled &
quantized
25 August 2011 1999-2011 by Richard Alan Peters II 12
Digital Image
a grid of squares,
each of which
contains a single
color
each square is
called a pixel (for
picture element)
Color images have 3 values per
pixel; monochrome images have
1 value per pixel.
25 August 2011 1999-2011 by Richard Alan Peters II 13
Color Images
Are constructed from three
intensity maps.
Each intensity map is pro-
jected through a color filter
(e.g., red, green, or blue, or
cyan, magenta, or yellow) to
create a monochrome image.
The intensity maps are
overlaid to create a color
image.
Each pixel in a color image is
a three element vector.
25 August 2011 1999-2011 by Richard Alan Peters II 14
Color
Images
On a
CRT
25 August 2011 1999-2011 by Richard Alan Peters II 15
Point Processing
original + gamma - gamma + brightness - brightness
original + contrast - contrast histogram EQ histogram mod
25 August 2011 1999-2011 by Richard Alan Peters II 16
Color Processing
requires some
knowledge of
how we see
colors
25 August 2011 1999-2011 by Richard Alan Peters II 17
Eyes Light Sensors
#(blue) << #(red) < #(green)
cone density near fovea
25 August 2011 1999-2011 by Richard Alan Peters II 18
Color Sensing / Color Perception
These are approximations
of the responses to the
visible spectrum of the
red, green, and blue
receptors of a typical
human eye.
25 August 2011 1999-2011 by Richard Alan Peters II 19
These are
approximations of
the responses to
the visible
spectrum of the
red, green,
and blue
receptors of a
typical human eye.
The simultaneous red + blue
response causes us to
perceive a continuous range
of hues on a circle. No hue is
greater than or less than any
other hue.
Color Sensing / Color Perception
25 August 2011 1999-2011 by Richard Alan Peters II 20
l
u
m
i
n
a
n
c
e

h
u
e

s
a
t
u
r
a
t
i
o
n

photo receptors brain
The eye has 3 types of photoreceptors:
sensitive to red, green, or blue light.
The brain transforms RGB into separate
brightness and color channels (e.g., LHS).
Color Sensing / Color Perception
25 August 2011 1999-2011 by Richard Alan Peters II 21
Color Perception
all bands luminance chrominance
red green blue
16 pixelization of:
luminance and chrominance
(hue+saturation) are perceived
with different resolutions, as
are red, green and blue.
25 August 2011 1999-2011 by Richard Alan Peters II 22
Color Perception
all bands luminance chrominance
red green blue
16 pixelization of:
25 August 2011 1999-2011 by Richard Alan Peters II 23
Color Balance
and Saturation
Uniform changes in color
components result in
change of tint.
E.g., if all G pixel values are
multiplied by > 1 then the
image takes a green cast.
25 August 2011 1999-2011 by Richard Alan Peters II 24
Color Transformations
222
222
218
222
222
185
17
122
114 236
227
106
240
171
103
240
171
160
17
121
171 240
230
166
17 17
122 121
114 171
222 222
222 222
185 218
240 240
171 171
103 160
236 240
227 230
106 166
Image aging: a transformation, , that mapped:
25 August 2011 1999-2011 by Richard Alan Peters II 25
The 2D Fourier Transform of a Digital Image
2 1 1
0 0
, , ,
ur vc
i R C
R C
u v
I r c u v e I
1 1 2
1
0 0
( , )
ur vc
R C i
R C
RC
r c
u,v I r c e I
Let I(r,c) be a single-band (intensity) digital image with R
rows and C columns. Then, I(r,c) has Fourier representation
where
are the R x C Fourier coefficients.
these complex
exponentials are
2D sinusoids.
25 August 2011 1999-2011 by Richard Alan Peters II 26
2D Sinusoids:
orientation
... are plane waves with
grayscale amplitudes,
periods in terms of lengths, ...
2
, cos cos sin 1
2 C R
A c r
I r c
A
= phase shift
r
c
25 August 2011 1999-2011 by Richard Alan Peters II 27
2D Sinusoids:
... specific orientations,
and phase shifts.
r
c
r
c
25 August 2011 1999-2011 by Richard Alan Peters II 28
The Value of a Fourier Coefficient
is a complex
number with a
real part and an
imaginary part.
If you represent
that number as a
magnitude, A, and
a phase, ,
..these represent the amplitude
and offset of the sinusoid with
frequency and direction .
25 August 2011 1999-2011 by Richard Alan Peters II 29
The Sinusoid from the Fourier Coeff. at (u,v)
25 August 2011 1999-2011 by Richard Alan Peters II 30
Here is the same coefficient plotted as
magnitude, A, and a phase, , and displayed
in the space domain as a sinusoid.
I
|F{I}| [F{I}]
The Fourier Transform of an Image
magnitude phase
25 August 2011 1999-2011 by Richard Alan Peters II 31
Continuous Fourier Transform
The continuous Fourier
transform assumes a
continuous image exists
in a finite region of an
infinite plane.
2 ( )
I , ,
i uc vr
r c u v e dudv I
2 ( )
, I ,
i uc vr
u v r c e dcdr I
The BoingBoing Bloggers
25 August 2011 1999-2011 by Richard Alan Peters II 32
Discrete Fourier Transform
The discrete Fourier
transform assumes a
digital image exists on a
closed surface, a torus.
1 1 2
0 0
I ( )
uc vr
R C i
C R
v u
r,c u,v e I
1 1
2
0 0
, I ,
cu rv
R C
i
C R
r c
u v r c e I
The BoingBoing Bloggers
25 August 2011 1999-2011 by Richard Alan Peters II 33
Convolution
16 , 16 c r
0 , 0 c r
16 , 16 c r 16 , 16 c r
16 , 16 c r
Sum times 1/5
Sums of shifted and
weighted copies of
images or Fourier
transforms.
25 August 2011 1999-2011 by Richard Alan Peters II 34
Convolution Property of the Fourier Transform
The Fourier Transform of a
product equals the
convolution of the Fourier
Transforms. Similarly, the
Fourier Transform of a
convolution is the product of
the Fourier Transforms
.
by computed be can n convolutio spatial a Then,
tion multiplica pointwise represents
n convolutio represents
. } {
Moreover,
. } {
Then,
). , ( and ) , ( Transforms Fourier
have ) , ( and ) , ( functions Let
1
G F g f
G F g f
G F g f
v u G v u F
c r g c r f
-
F
F
F
25 August 2011 1999-2011 by Richard Alan Peters II 35
Sampling, Aliasing, & Frequency Convolution
aliasing (the jaggies) no aliasing (smooth lines)
25 August 2011 1999-2011 by Richard Alan Peters II 36
Sampling,
Aliasing, &
Frequency
Convolution
(a) (b)
(c) (d)
(a) aliased
(b) power spectrum
(c) unaliased
(d) power spectrum
25 August 2011 1999-2011 by Richard Alan Peters II 37
8 16
Resampling
nearest neighbor nearest neighbor
bicubic interpolation bicubic interpolation
(resizing)
25 August 2011 1999-2011 by Richard Alan Peters II 38
Rotation
a n d m o t i o n b l u r

25 August 2011 1999-2011 by Richard Alan Peters II 39
Image Warping
25 August 2011 1999-2011 by Richard Alan Peters II 40
Panorama via Overlay
Originals
Merged
*

*
not so good.
B
r
u
n
o

P
o
s
t
l
e

h
t
t
p
:
/
/
h
u
g
i
n
.
s
o
u
r
c
e
f
o
r
g
e
.
n
e
t
/
t
u
t
o
r
i
a
l
s
/
t
w
o
-
p
h
o
t
o
s
/
e
n
.
s
h
t
m
l

25 August 2011 1999-2011 by Richard Alan Peters II 41
Panorama via Stitching
Originals
Merged
*

*
much better.
B
r
u
n
o

P
o
s
t
l
e

h
t
t
p
:
/
/
h
u
g
i
n
.
s
o
u
r
c
e
f
o
r
g
e
.
n
e
t
/
t
u
t
o
r
i
a
l
s
/
t
w
o
-
p
h
o
t
o
s
/
e
n
.
s
h
t
m
l

25 August 2011 1999-2011 by Richard Alan Peters II 42
Gaussian LPF in FD Original Image Power Spectrum
Image size: 512x512
SD filter sigma = 8 Frequency Domain (FD) Filtering
25 August 2011 1999-2011 by Richard Alan Peters II 43
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512
SD filter sigma = 8 FD Filtering: Lowpass
25 August 2011 1999-2011 by Richard Alan Peters II 44
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512
FD notch sigma = 8
FD Filtering: Highpass
25 August 2011 1999-2011 by Richard Alan Peters II 45
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512
FD notch sigma = 8
FD Filtering: Highpass
signed image with
0 at middle gray
25 August 2011 1999-2011 by Richard Alan Peters II 46
original blurred sharpened
Spatial Filtering
25 August 2011 1999-2011 by Richard Alan Peters II 47
Spatial Filtering
bandpass
filter
unsharp
masking
original
25 August 2011 1999-2011 by Richard Alan Peters II 48
Spatial Filtering
bandpass
filter
unsharp
masking
original
signed image with
0 at middle gray
25 August 2011 1999-2011 by Richard Alan Peters II 49
Motion Blur
vertical regional
zoom
rotational
original
25 August 2011 1999-2011 by Richard Alan Peters II 50
color noise blurred image color-only blur
Noise Reduction
25 August 2011 1999-2011 by Richard Alan Peters II 51
5x5 Wiener filter color noise blurred image
Noise Reduction
25 August 2011 1999-2011 by Richard Alan Peters II 52
Noise Reduction
original
periodic
noise
frequency
tuned filter
25 August 2011 1999-2011 by Richard Alan Peters II 53
Shot Noise or Salt & Pepper Noise
+ shot noise - shot noise s&p noise
25 August 2011 1999-2011 by Richard Alan Peters II 54
Nonlinear Filters: the Median
s&p noise original median filter
25 August 2011 1999-2011 by Richard Alan Peters II 55
Nonlinear Filters: Min and Maxmin
+ shot noise min filter maxmin filter
25 August 2011 1999-2011 by Richard Alan Peters II 56
Nonlinear Filters: Max and Minmax
- shot noise max filter minmax
25 August 2011 1999-2011 by Richard Alan Peters II 57
Nonlinear Processing: Binary Morphology
L shaped SE
O marks origin
Foreground: white pixels
Background: black pixels
Cross-hatched
pixels are
indeterminate.
25 August 2011 1999-2011 by Richard Alan Peters II 58
Used after opening to grow back pieces of the original
image that are connected to the opening.
Permits the removal of small regions that are disjoint
from larger objects without distorting the small
features of the large objects.
original opened reconstructed
Nonlinear Processing: Binary Reconstruction
25 August 2011 1999-2011 by Richard Alan Peters II 59
L shaped SE
O marks origin
Foreground: white pixels
Background: black pixels
Cross-hatched
pixels are
indeterminate.
Nonlinear Processing: Grayscale Morphology
25 August 2011 1999-2011 by Richard Alan Peters II 60
Grayscale Morphology: Opening
opening: erosion then dilation opened & original
25 August 2011 1999-2011 by Richard Alan Peters II 61
Grayscale Morphology: Opening
erosion & opening erosion & opening & original
25 August 2011 1999-2011 by Richard Alan Peters II 62
reconstructed opening original
Nonlinear Processing: Grayscale Reconstruction
25 August 2011 1999-2011 by Richard Alan Peters II 63
Forensic Analysis of Photographs
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855.
From Morris, Errol, Which Came First, the Chicken or the Egg?, Parts 1-3, New York
Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Which came first?
25 August 2011 1999-2011 by Richard Alan Peters II 64
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, Which Came First, the
Chicken or the Egg?, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Which came first?
Forensic Analysis of Photographs
25 August 2011 1999-2011 by Richard Alan Peters II 65
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, Which Came First, the
Chicken or the Egg?, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Forensic Analysis of Photographs
Which came first?
25 August 2011 1999-2011 by Richard Alan Peters II 66
High Dynamic Range
(HDR) Imaging
under exposed
B
a
r
t
l
o
m
i
e
j

O
k
o
n
e
k


h
t
t
p
:
/
/
w
w
w
.
e
a
s
y
h
d
r
.
c
o
m
/
e
x
a
m
p
l
e
s
.
p
h
p

25 August 2011 1999-2011 by Richard Alan Peters II 67
High Dynamic Range
(HDR) Imaging
default exposure
B
a
r
t
l
o
m
i
e
j

O
k
o
n
e
k


h
t
t
p
:
/
/
w
w
w
.
e
a
s
y
h
d
r
.
c
o
m
/
e
x
a
m
p
l
e
s
.
p
h
p

25 August 2011 1999-2011 by Richard Alan Peters II 68
High Dynamic Range
(HDR) Imaging
over exposed
B
a
r
t
l
o
m
i
e
j

O
k
o
n
e
k


h
t
t
p
:
/
/
w
w
w
.
e
a
s
y
h
d
r
.
c
o
m
/
e
x
a
m
p
l
e
s
.
p
h
p

25 August 2011 1999-2011 by Richard Alan Peters II 69
High Dynamic Range
(HDR) Imaging
combined
B
a
r
t
l
o
m
i
e
j

O
k
o
n
e
k


h
t
t
p
:
/
/
w
w
w
.
e
a
s
y
h
d
r
.
c
o
m
/
e
x
a
m
p
l
e
s
.
p
h
p

25 August 2011 1999-2011 by Richard Alan Peters II 70
Image Compression
Yoyogi Park, Tokyo, October 1999. Photo by Alan Peters.
Original image is
5244w x 4716h
@ 1200 ppi:
127MBytes
25 August 2011 1999-2011 by Richard Alan Peters II 71
Image Compression: JPEG
J
P
E
G

q
u
a
l
i
t
y

l
e
v
e
l

F
i
l
e

s
i
z
e

i
n

b
y
t
e
s

25 August 2011 1999-2011 by Richard Alan Peters II 72
J
P
E
G

q
u
a
l
i
t
y

l
e
v
e
l

F
i
l
e

s
i
z
e

i
n

b
y
t
e
s

Image Compression: JPEG
25 August 2011 1999-2011 by Richard Alan Peters II 73
Image Compositing
Combine parts from separate images to form a new image.
Its difficult to do well.
Requires relative positions, orientations, and scales to be
correct.
Lighting of objects must be consistent within the separate
images.
Brightness, contrast, color balance, and saturation must
match.
Noise color, amplitude, and patterns must be seamless.
25 August 2011 1999-2011 by Richard Alan Peters II 74
Prof. Peters in his home office. Needs a better shirt.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 75
This shirt demands a monogram.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 76
He needs some more color.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 77
Nice. Now for the way hed wear his hair if he had any.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 78
He cant stay in the office like this.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 79
Wheres a hepcat Daddy-O like this belong?
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 80
In the studio!
Collar this jive,
Jackson. Like
crazy, Man !
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 81