Eece253 01 Intro

EECE/CS 253 Image Processing
Lecture Notes: Introduction and Overview
Richard Alan Peters II

Department of Electrical Engineering and
Computer Science
Fall Semester 2007
This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or
send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.
Introduction and Overview
This presentation is an
overview of some of the
ideas and techniques to be
covered during the course.
26 April 2008 1999-2007 by Richard Alan Peters II 2

Topics
Topics
1. image formation
2. point processing and equalization
3. color correction
4. the fourier transform
5. convolution
6. image sampling and warping
7. spatial filtering
8. noise reduction
9. mathematical morphology
10. image compression

Wallace and Gromit
Wallace
readsElectronics
reads Electronics for
for Dogs
Dogs
Gromit likes cheese

cheese
likes
Wallace and Gromit will be subjects of some of the imagery in this introduction.
http://www.aardman.com/wallaceandgromit/index.shtml

lens
Image
Image Formation
Formation
t
jec
ob
image
plane

Image
Image Formation
Formation
ce
ur
so
ht
lig

Image
Image Formation
Formation
projection
projection
through
through lens
lens
image
image of
of object
object

Image
Image Formation
Formation
projection
projection onto
onto
discrete
discrete sensor
sensor digital
digital camera
camera
array.
array.

Image
Image Formation
Formation
sensors
sensors register
register
average
average color.
color. sampled
sampled image
image

Image
Image Formation
Formation
continuous
continuous colors,
colors,
discrete
discrete locations.
locations. discrete
discrete real-
real-
valued
valued image
image

Digital Image Formation: Quantization
discrete color output
continuous
continuouscolors
colors
mapped
mappedto toaafinite,
finite,
discrete
discreteset
setof
ofcolors.
colors.
continuous color input

Sampling and Quantization
pixel grid
real image sampled quantized sampled &

quantized

Color images have 3 values per
Digital Image pixel; monochrome images have
1 value per pixel.
aagrid
gridof
ofsquares,
squares,
eachof
each ofwhich
which
containsaasingle
contains single
color
color
eachsquare
each squareisis
calledaapixel
called pixel(for
(for
pictureelement)
picture element)

Color Images
O Are constructed from three
intensity maps.
O Each intensity map is pro-
jected through a color filter
(e.g., red, green, or blue, or
cyan, magenta, or yellow) to
create a monochrome image.
O The intensity maps are
overlaid to create a color
image.
O Each pixel in a color image is
a three element vector.

Color
Color
Images
Images
On
On aa
CRT
CRT

Point
Point Processing
Processing
- gamma - brightness original + brightness + gamma
histogram mod - contrast original + contrast histogram EQ

Color Processing
requires
requiressome
some
knowledge
knowledgeofof
how
howwewesee
see
colors
colors

Eye’s Light Sensors
cone density near fovea
#(blue) << #(red) < #(green)

Color Sensing / Color Perception
These
Theseareareapproximations
approximations
of
of the responsestotothe
the responses the
visible spectrum of the
visible spectrum of the
“red”,
“red”,“green”,
“green”,and
and“blue”
“blue”
receptors of a typical
receptors of a typical
human
humaneye.eye.

These
Theseare
are
approximations
approximationsof of
the responses to
the responses to
the
thevisible
visible
spectrum of the
spectrum of the
“red”,
“red”,“green”,
“green”,
and
and“blue”
“blue”
receptors
receptorsof ofaa
typical
typicalhuman
humaneye.
eye.
The
Thesimultaneous
simultaneousredred++blue
blue
response causes us to
response causes us to
perceive
perceiveaacontinuous
continuousrange
range
of
ofhues
hueson
onaacircle.
circle. No
Nohue
hueisis
greater
greaterthan
thanororless
lessthan
thanany
any
other
otherhue.
hue.

The eye has 3 types of photoreceptors:
luminance
sensitive to red, green, or blue light.

hue
saturation
The brain transforms RGB into separate

brightness and color channels (e.g., LHS).
brain photo receptors

luminanceand
andchrominance
chrominance
Color Perception luminance
(hue+saturation)are
(hue+saturation) areperceived
perceived
16× pixelization of: withdifferent
with differentresolutions,
resolutions,as
as
arered,
are red,green
greenand
andblue.
blue.
all
allbands
bands luminance
luminance chrominance
chrominance
red
red green
green blue
blue

Color Perception
16× pixelization of:
all
allbands
bands luminance
luminance chrominance
chrominance
red
red green
green blue
blue

Color Balance
and Saturation
Uniform changes in color
components result in
change of tint.
E.g., if all G pixel values are

multiplied by α > 1 then the
image takes a green cast.

Color Transformations
⎡222⎤ ⎡222⎤
⎢222⎥ ⎢222⎥
⎢ ⎥ ⎢ ⎥
⎣218⎦ ⎣185 ⎦
⎡ 17 ⎤ ⎡ 17 ⎤
⎢121⎥ ⎢122⎥
⎢ ⎥ ⎡240⎤ ⎢ ⎥ ⎡236⎤
⎣171⎦ ⎢230⎥ ⎣114⎦ ⎢227⎥
⎢ ⎥ ⎢ ⎥
⎣166 ⎦ ⎣106 ⎦
⎡240⎤ ⎡240⎤
⎢171⎥ ⎢171⎥
⎢ ⎥ ⎢ ⎥
⎣160 ⎦ ⎣103⎦
Image aging: a transformation, Φ, that mapped:

⎡ 17 ⎤ ⎧⎡ 17 ⎤ ⎫ ⎡222⎤ ⎧⎡222⎤ ⎫ ⎡240⎤ ⎧⎡240⎤ ⎫ ⎡236⎤ ⎧⎡240⎤ ⎫
⎢122⎥ = Φ ⎪⎨⎢121⎥ ⎪⎬ ⎢222⎥ = Φ ⎪⎨⎢222⎥ ⎪⎬ ⎢171⎥ = Φ ⎪⎨⎢171⎥ ⎪⎬ ⎢227⎥ = Φ ⎪⎨⎢230⎥ ⎪⎬
⎢ ⎥ ⎪⎩⎢⎣171⎥⎦ ⎪⎭ ⎢ ⎥ ⎪⎩⎢⎣218⎥⎦ ⎪⎭ ⎢ ⎥ ⎪⎩⎢⎣160 ⎥⎦ ⎪⎭ ⎢ ⎥ ⎪⎩⎢⎣166 ⎥⎦ ⎪⎭
⎣114⎦ ⎣185 ⎦ ⎣103⎦ ⎣106 ⎦

The 2D Fourier Transform of a Digital Image
Let I(r,c) be a single-band (intensity) digital image with R
rows and C columns. Then, I(r,c) has Fourier representation
⎛ ur vc ⎞
R −1 C −1 + i 2π ⎜ + ⎟
I ( r , c ) = ∑ ∑ I ( u ,v ) e ⎝R C⎠
, these complex
complex
u =0 v =0 these
exponentials are
exponentials are
where
2D sinusoids.
2D sinusoids.
⎛ ur vc ⎞
R −1 C −1 − i 2π ⎜ + ⎟
I ( u,v ) = RC1 ∑ ∑ I ( r,c) e ⎝R C⎠
r =0 c =0
are the R x C Fourier coefficients.

A ⎧ ⎡ 2π ⎛ c r ⎞ ⎤ ⎫
2D Sinusoids: I (r, c ) = ⎨cos ⎜ cos θ − sin θ ⎟ + φ ⎥ + 1⎬
2 ⎩ ⎢⎣ λ ⎝ C R ⎠ ⎦ ⎭
c
...
...are
areplane
planewaves
waveswith
with
grayscale
grayscaleamplitudes,
amplitudes,
periods
periodsininterms
termsof
oflengths,
lengths,...
... θ
r
orientation
φ = phase shift

2D Sinusoids: ...
...specific
specificorientations,
and
andphase
orientations,
phaseshifts.
shifts.
c c
a t ion
r nt r
orie

The Value of a Fourier Coefficient …
……isisaacomplex
complex
numberwith
number withaa
realpart
real partand
andanan
imaginarypart.
imaginary part.
Ifyou
If yourepresent
represent
thatnumber
that numberas asaa
magnitude,A,
magnitude, and
A,and
aaphase,
phase,φ,φ,……
..theserepresent
..these representthe
theamplitude
amplitude
andoffset
and offsetof
ofthe
thesinusoid
sinusoidwith
with
frequencyωωand
frequency anddirection
directionθ.θ.

The Sinusoid from the Fourier Coeff. at (u,v)

The Fourier Transform of an Image
magnitude phase
I |F{I}| ∠[F{I}]

Continuous Fourier Transform
∞ ∞
I(r, c ) = ∫ ∫ I(u, v ) e
+i 2π ( uc + vr )
dudv
− ∞− ∞
∞ ∞
I(u, v ) = ∫ ∫ I (r , c ) e −i 2π ( uc + vr )
dcdr
− ∞− ∞
Thecontinuous
The continuousFourier
Fourier
transformassumes
transform assumesaa
continuousimage
continuous imageexists
exists
ininaafinite
finiteregion
regionof
ofan
an
infiniteplane.
infinite plane.
The BoingBoing
The BoingBoing Bloggers
Bloggers

Discrete Fourier Transform
Thediscrete
The discreteFourier
Fourier
transformassumes
transform assumesaa
digitalimage
digital imageexists
existson
onaa
closedsurface,
closed surface,aatorus.
torus.
⎛ uc vr ⎞
R − 1C − 1 + i 2π ⎜ + ⎟
I( r,c ) = ∑ ∑ I(u,v ) e ⎝C R ⎠
v= 0 u= 0
⎛ cu rv ⎞
R −1C −1 − i 2π ⎜ + ⎟
I(u, v ) = ∑ ∑ I(r,c ) e ⎝C R ⎠
r =0 c =0
The BoingBoing Bloggers

Convolution Sums of shifted and
weighted copies of
images or Fourier
transforms.
δ (r + 16, c + 16) δ (r + 16, c − 16)
δ (r − 0, c − 0)
δ (r − 16, c + 16) δ (r − 16, c − 16)
Sumtimes
Sum times1/5
1/5

Convolution Property of the Fourier Transform
Let functions f ( r, c ) and g ( r, c ) have

Fourier Transforms F (u, v ) and G (u, v ).
TheFourier
The FourierTransform
Transformof ofaa
productequals
product equalsthe the
Then,
convolutionof
convolution ofthetheFourier
Fourier
F { f ∗ g } = F ⋅ G. Transforms. Similarly,
Transforms. Similarly, the
the
Moreover, FourierTransform
Fourier Transformof ofaa
F { f ⋅ g } = F ∗ G. convolutionisisthe
convolution theproduct
productof of
∗ represents convolutio n theFourier
the FourierTransforms
Transforms
⋅ represents pointwise multiplica tion
Then, a spatial convolution can be computed by
f ∗g =F
-1
{F ⋅ G}.

Sampling, Aliasing, & Frequency Convolution
aliasing (the jaggies) no aliasing (smooth lines)

Sampling,
Aliasing, &
Frequency
Convolution
(a) (b)
(a)
(a) aliased
aliased
(b)
(b) power
powerspectrum
spectrum
(c)
(c) unaliased
unaliased
(d)
(d) power
powerspectrum
spectrum (c) (d)

Resampling
nearest
nearest neighbor
neighbor nearest
nearest neighbor
neighbor
8× 16×
(resizing)
bicubic
bicubic interpolation
interpolation bicubic
bicubic interpolation
interpolation

Rotation
and motion blur

Image Warping

Imagesize:
Image size:512x512
512x512
Frequency Domain (FD) Filtering SDfilter
SD filtersigma
sigma==88
OriginalImage
Original Image PowerSpectrum
Power Spectrum GaussianLPF
Gaussian LPFininFD
FD

Imagesize:
size:512x512
512x512
FD Filtering: Lowpass Image
SDfilter
SD filtersigma
sigma==88
FilteredImage
Filtered Image FilteredPower
Filtered PowerSpectrum
Spectrum OriginalImage
Original Image

Imagesize:
size:512x512
512x512
FD Filtering: Highpass Image
FDnotch
FD notchsigma
sigma==88
FilteredImage
Original Image

Imagesize:
size:512x512
512x512
FD Filtering: Highpass Image
FDnotch
FD notchsigma
sigma==88
signedimage
signed imagewith
with
00at
atmiddle
middlegray
gray
FilteredImage
Original Image

Spatial Filtering
blurred original sharpened

Spatial Filtering
bandpass original unsharp

filter masking

Spatial Filtering
signed
signedimage
imagewith
with
00at
atmiddle
middlegray
gray
bandpass original unsharp

filter masking

Motion Blur
regional vertical
original
zoom rotational

Noise Reduction
blurred image color noise color-only blur

Noise Reduction
blurred image color noise 5x5 Wiener filter

Noise Reduction
periodic frequency
original
noise tuned filter
Shot Noise or Salt & Pepper Noise
+ shot noise s&p noise - shot noise

Nonlinear Filters: the Median
original s&p noise median filter

Nonlinear Filters: Min and Maxmin
+ shot noise min filter maxmin filter

Nonlinear Filters: Max and Minmax
- shot noise max filter minmax

Nonlinear Processing: Binary Morphology
“L” shaped SE Foreground: white pixels Cross-hatched

pixels are
O marks origin Background: black pixels indeterminate.

Nonlinear Processing: Binary Reconstruction
O Used after opening to grow back pieces of the original
image that are connected to the opening.
O Permits the removal of small regions that are disjoint
from larger objects without distorting the small
features of the large objects.
original opened reconstructed

Nonlinear Processing: Grayscale Morphology
“L” shaped SE Foreground: white pixels Cross-hatched

pixels are
O marks origin Background: black pixels indeterminate.

Grayscale Morphology: Opening
opening: erosion then dilation opened & original

Grayscale Morphology: Opening
erosion & opening erosion & opening & original

Nonlinear Processing: Grayscale Reconstruction
original reconstructed opening

Forensic Analysis of Photographs
Which came first?
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855.
From Morris, Errol, “Which Came First, the Chicken or the Egg?”, Parts 1-3, New York
Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Which
Whichcame
camefirst?
first?
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the
Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Which
Whichcame
camefirst?
first?
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the
Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Image Compression
Original
Originalimage
imageisis
5244w
5244wxx4716h
4716h
@@1200
1200ppi:
ppi:
127MBytes
127MBytes
Yoyogi Park, Tokyo, October 1999. Photo by Alan Peters.

JPEG quality level Image Compression: JPEG
File size in bytes

Image Compression: JPEG
JPEG quality level
File size in bytes

Image Compositing
O Combine parts from separate images to form a new image.
O It’s difficult to do well.
O Requires relative positions, orientations, and scales to be
correct.
O Lighting of objects must be consistent within the separate
images.
O Brightness, contrast, color balance, and saturation must
match.
O Noise color, amplitude, and patterns must be seamless.

Image Compositing Example
Prof. Peters in his home office. Needs a better shirt.

This shirt demands a monogram.

He needs some more color.

Nice. Now for the way he’d wear his hair if he had any.

He can’t stay in the office like this.

Where’s a hepcat Daddy-O like this belong?

Collar
Collarthis
thisjive,
jive,
Jackson.
Jackson. Like
Like
crazy
crazy,,Man
Man! !
In the studio!

Eece253 01 Intro

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Eece253 01 Intro

Transféré par

Droits d'auteur :

Formats disponibles

EECE/CS 253 Image Processing

Lecture Notes: Introduction and Overview

Richard Alan Peters II

26 April 2008 1999-2007 by Richard Alan Peters II 2

26 April 2008 1999-2007 by Richard Alan Peters II 3

Gromit likes cheese

26 April 2008 1999-2007 by Richard Alan Peters II 4

26 April 2008 1999-2007 by Richard Alan Peters II 5

26 April 2008 1999-2007 by Richard Alan Peters II 6

26 April 2008 1999-2007 by Richard Alan Peters II 7

26 April 2008 1999-2007 by Richard Alan Peters II 8

26 April 2008 1999-2007 by Richard Alan Peters II 9

26 April 2008 1999-2007 by Richard Alan Peters II 10

continuous color input

26 April 2008 1999-2007 by Richard Alan Peters II 11

real image sampled quantized sampled &

26 April 2008 1999-2007 by Richard Alan Peters II 12

26 April 2008 1999-2007 by Richard Alan Peters II 13

26 April 2008 1999-2007 by Richard Alan Peters II 14

26 April 2008 1999-2007 by Richard Alan Peters II 15

- gamma - brightness original + brightness + gamma

histogram mod - contrast original + contrast histogram EQ

26 April 2008 1999-2007 by Richard Alan Peters II 16

26 April 2008 1999-2007 by Richard Alan Peters II 17

#(blue) << #(red) < #(green)

26 April 2008 1999-2007 by Richard Alan Peters II 18

26 April 2008 1999-2007 by Richard Alan Peters II 19

26 April 2008 1999-2007 by Richard Alan Peters II 20

sensitive to red, green, or blue light.

The brain transforms RGB into separate

26 April 2008 1999-2007 by Richard Alan Peters II 21

26 April 2008 1999-2007 by Richard Alan Peters II 22

26 April 2008 1999-2007 by Richard Alan Peters II 23

E.g., if all G pixel values are

26 April 2008 1999-2007 by Richard Alan Peters II 24

Image aging: a transformation, Φ, that mapped:

26 April 2008 1999-2007 by Richard Alan Peters II 25

are the R x C Fourier coefficients.

26 April 2008 1999-2007 by Richard Alan Peters II 26

26 April 2008 1999-2007 by Richard Alan Peters II 27

26 April 2008 1999-2007 by Richard Alan Peters II 28

26 April 2008 1999-2007 by Richard Alan Peters II 29

26 April 2008 1999-2007 by Richard Alan Peters II 30

26 April 2008 1999-2007 by Richard Alan Peters II 31

26 April 2008 1999-2007 by Richard Alan Peters II 32

The BoingBoing Bloggers

26 April 2008 1999-2007 by Richard Alan Peters II 33

δ (r + 16, c + 16) δ (r + 16, c − 16)

δ (r − 16, c + 16) δ (r − 16, c − 16)

26 April 2008 1999-2007 by Richard Alan Peters II 34

Let functions f ( r, c ) and g ( r, c ) have

26 April 2008 1999-2007 by Richard Alan Peters II 35

aliasing (the jaggies) no aliasing (smooth lines)

26 April 2008 1999-2007 by Richard Alan Peters II 36

26 April 2008 1999-2007 by Richard Alan Peters II 37

26 April 2008 1999-2007 by Richard Alan Peters II 38

and motion blur

26 April 2008 1999-2007 by Richard Alan Peters II 39

26 April 2008 1999-2007 by Richard Alan Peters II 40

26 April 2008 1999-2007 by Richard Alan Peters II 41

26 April 2008 1999-2007 by Richard Alan Peters II 42

26 April 2008 1999-2007 by Richard Alan Peters II 43

26 April 2008 1999-2007 by Richard Alan Peters II 44

blurred original sharpened