Computer

Vision

Recording
and Displaying
of Images

Computer
Vision

Acquisition and display of images

A readers digest :
1. displays
2. cameras
3. illumination

Computer
Vision

displays

Computer
Vision

Cathode Ray Tubes

Computer
Vision

The video standard

viewing quality psychophysics
discrete character, both spatial and temporal,
should be invisible
video standard defines :
1. number of lines and columns
2. aspect ratio
3. scanning frequency

Computer
Vision

TV : number of lines

Aspect ratio of pixels = 4 /3

Europe : CCIR : 625 lines, 575 visible
USA : RETMA : 525 lines, 484 visible
(other lines lost during flyback)

Computer
Vision

TV : sampling frequency
flickerfree images : at least 60 Hz
halved by interlacing : odd and even lines
separately :
1st field
2nd
field
1
2
3
4
...
5
...

E
T
IN

Computer
Vision

G
N
I
C
A
RL

Europe : repetition of 25Hz (50/2)

USA : 30Hz (60/2)

LCD display
Liquid Crystal Display
polarizers

passive pixel

active pixel
No light generation use backlighting

Computer
Vision

LCD display
Liquid Crystal Display
polarizers

passive pixel
Electrical field

active pixel
No light generation use backlighting

Computer
Vision

cameras

Computer
Vision

Optics for image formation

the pinhole model :

X i Yi
f
= =
= m
X o Yo Z o

Computer
Vision

(m = linear magnification)

Camera obscura + lens

Computer
Vision

The thin-lens equation

lens to capture enough light :

1
1 1
=
ZO Zi f
PO

Computer
Vision

assuming
spherical lens surfaces
incoming light parallel to axis
thickness << radii
same refractive index on both sides

The depth-of-field

Z 0 = Z 0 Z 0 =

Z 0 (Z 0 f )
Z0 + f d / b f

decreases with d, increases with Z0

strike a balance between incoming light and
large depth range

Computer
Vision

Deviations from the lens model

3 assumptions :
1. all rays from a point are focused onto 1 image point
2. all image points in a single plane
3. magnification is constant
deviations from this ideal are aberrations

Computer
Vision

Aberrations
2 types :

1. geometrical
2. chromatic
geometrical : small for paraxial rays
chromatic : refractive index function of
wavelength

Computer
Vision

Geometrical aberrations
spherical aberration
astigmatism

the most important type

radial distortion
coma

aberrations are reduced by combining lenses

Computer
Vision

Radial Distortion
magnification different
for different angles of inclination

Can be corrected if parameters are known

Computer
Vision

Chromatic aberration
rays of different wavelengths focused
in different planes

cannot be removed completely

sometimes achromatization is achieved for
more than 2 wavelengths

Computer
Vision

Cameras

we consider 2 types :

1. CCD
2. CMOS

10

Computer
Vision

Cameras

Computer
Vision

Cameras

11

Computer
Vision

Cameras

Computer
Vision

The CCD camera

12

Computer
Vision

CMOS
Same sensor elements as CCD
Each photo sensor has its own amplifier
More noise (reduced by subtracting black image)
Lower sensitivity (lower fill rate)

Uses standard CMOS technology

Allows to put other components on chip
Smart pixels

Foveon
4k x 4k sensor
0.18 process
70M transistors

Computer
Vision

CCD vs. CMOS

Mature technology
Specific technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout

Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration
with other components

2006 was year of sales cross-over

13

Computer
Vision

Colour cameras

We consider 3 concepts:
1. Prism (with 3 sensors)
2. Filter mosaic
3. Filter wheel

Computer
Vision

Prism colour camera

Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation

14

Computer
Vision

Prism colour camera

Computer
Vision

Filter mosaic
Coat filter directly on sensor

Demosaicing (obtain full colour & full resolution image)

15

Computer
Vision

Filter wheel
Rotate multiple filters in front of lens
Allows more than 3 colour bands

Only suitable for static scenes

Computer
Vision

Prism vs. mosaic vs. wheel

approach
# sensors
Separation
Cost
Framerate
Artefacts
Bands

Prism
3
High
High
High
Low
3

Mosaic
1
Average
Low
High
Aliasing
3

Wheel
1
Good
Average
Low
Motion
3 or more

High-end
cameras

Low-end
cameras

Scientific
applications

16

Computer
Vision

Upcoming X3 technology of Foveon

Exploits the wavelength
dependent depth to
which a photon
penetrates silicon
And splits colors without
the use of any filters

creates a stack of pixels at one place

new CMOS technology

Computer
Vision

Geometric camera model

perspective projection

(Man Drawing a Lute, woodcut, 1525, Albrecht Drer)

17

Computer
Vision

Models for camera projection

the pinhole model revisited :

center of the lens = center of projection

notice the virtual image plane
this is called perspective projection

Computer
Vision

Perspective projection
Zc

u
v

Xc
Yc
origin lies at the center of projection

Y
X
theuZ =
axisf coincides withvthe
=optical
f axis
X -axis to image
Z rows, Y -axis Zto columns
c

18

Computer
Vision

Pseudo-orthographic projection

u= f

X
Z

v= f

Y
Z

If Z is constant x= kX and y = kY,

where k=f/Z
i.e. orthographic projection + a scaling

Good approximation if /Z constant, i.e. if objects

are small compared to their distance from the camera

Computer
Vision

Pictoral comparison
Pseudo orthographic

Perspective

19

Computer
Vision

Projection matrices
the perspective projection model is incomplete :
what if :
1. 3D coordinates are specified in a
world coordinate frame
2. Image coordinates are expressed as
row and column numbers
We will not consider additional refinements,
such as radial distortions,...

Computer
Vision

(X,Y,Z)

Projection
matrices
(u,v)

u= f

r2

v= f

u
X

r3

r1

r1,P C
r3,P C
r2 ,P C
r3,P C

u= f

r11 ( X C1 ) + r12 (Y C2 ) + r13 ( Z C3 )

r31 ( X C1 ) + r32 (Y C2 ) + r33 ( Z C3 )

v= f

r21 ( X C1 ) + r22 (Y C2 ) + r23 ( Z C3 )

r31 ( X C1 ) + r32 (Y C2 ) + r33 ( Z C3 )

20

Computer
Vision

Projection matrices
Image coordinates are to be expressed as
pixel coordinates
x

012

0
1
2
3

x = k x u + s v + x0

k y v + y0
y =
with :

Computer
Vision

(x0, y0) the pixel coordinates of the principal point

NB7
kx the knumber
ofand
pixels
percalled
unit
length
horizontally
means
internally
NB6
NB4
they
are
are
known,
the
the
camera
camera
is
NB3
,s,x
ythe
are
internal
camera
xy,k
0 and
0known,
NB1::::fully
often
only
integer
pixel
coordinates
matter
NB2
kwhen
/kyxcalibrated
isthese
called
aspect
ratio
NB5
vector
C
matrix
R

SO
(3)and
areis
the
externally
kparameters
of pixels per unit length vertically
y the number
calibrated
externally
internally
calibrated
external camera parameters
s indicates the skew ; typically s = 0

Projection matrices
Exploiting homogeneous coordinates :

xu k x fs r11x0

yv= =0 kf y r21y0
1
1 0 0r311

fr12f 0f r013r11Xr12 C
r131 X C1

0 ff r023r21Y r22 Cr23

Y
C

fr22
2
2
0 0 1 r r r Z C
r3331Z 32 C333
3
r32

We
have
Wealso
define
the calibration matrix :

k xx s x0k xsf

K = 0 y k y= y00 k0y
1 0
0 0 10

0x0 0u f k x f s x0

fy0 0v= 0 f k y y0

01 11 0
0 1

21

Projection matrices

Computer
Vision

We define

x

p = y ;
1

X

P = Y ,
Z

X

~ Y
P =
Z

1

yielding

p = KR t ( P C )
or,
or,

for some non-zero

p = K (R t | R t C )P
~
p = ( M | t ) P with rank M = 3
~

Computer
Vision

From object radiance to pixel grey levels

After the geometric camera model...
a

camera model

2 steps:
1. from object radiance to image irradiance
2. from image irradiance to pixel grey level

22

Computer
Vision

Image irradiance and object radiance

we look at the irradiance that an object patch
will cause in the image
assumptions :
radiance R assumed known and
object at large distance compared to the focal length

Is image irradiance directly related to the radiance

of the image patch?

Computer
Vision

The viewing conditions

I=

AA
F
= R 0 2l cos 3 cos
Ai
Ai Z

=R

Al
cos4
2
f

23

Computer
Vision

The cos4 law contd

Especially strong effects

for wide-angle and
fisheye lenses

Computer
Vision

From irradiance to gray levels

f = g I + d
Gain
gamma
Dark reference

24

Computer
Vision

illumination

Computer
Vision

Illumination
Well-designed illumination often is key in
visual inspection

25

Computer
Vision

Illumination techniques
Simplify the image processing by controlling
the environment

An overview of illumination techniques:

1. back-lighting
2. directional-lighting
3. diffuse-lighting
4. polarized-lighting
5. coloured-lighting
6. structured-lighting
7. stroboscopic lighting

Computer
Vision

Back-lighting

lamps placed behind a transmitting diffuser plate,

light source behind the object
generates high-contrast silhouette images,
easy to handle with binary vision
often used in inspection

26

Computer
Vision

Example backlighting

Computer
Vision

Directional and diffuse lighting

Directional-lighting
generate sharp shadows
generation of specular reflection
(e.g. crack detection)
shadows and shading yield information about
shape
Diffuse-lighting
illuminates uniformly from all directions
prevents sharp shadows and large intensity
variations over glossy surfaces

27

Computer
Vision

Crack detection

Computer
Vision

Example directional lighting

28

Computer
Vision

Polarized lighting
2 uses:

1. to improve contrast between Lambertian and

specular reflections
2. to improve contrasts between dielectrics and
metals

Computer
Vision

Polarized lighting
specular reflection keeps polarisation :
diffuse reflection depolarises
suppression of specular reflection :

polarizer/analyzer crossed
prevents the large dynamic range caused by glare

29

Computer
Vision

Example pol. lighting (pol./an.crossed)

Computer
Vision

Polarised lighting
distinction between specular reflection from
dielectrics and metals;
works under the Brewster angle for the dielectric
dielectric has no parallel comp. ; metal does
suppression of specular reflection from dielectrics :

polarizer/analyzer aligned
distinguished metals and dielectrics

30

Computer
Vision

Example pol. lighting (pol./an. aligned)

Computer
Vision

Coloured lighting
highlight regions of a similar colour
with band-pass filter: only light from projected pattern
(e.g. monochromatic light from a laser)
differentiation between specular and diffuse reflection
comparing colours ; same spectral composition of
sources!
spectral sensitivity function of the sensors!

31

Computer
Vision

Example coloured lighting

Computer
Vision

Structured and stroboscopic lighting

spatially or temporally modulated light pattern
Structured lighting
e.g. : 3D shape : objects distort the projected
pattern
(more on this later)
Stroboscopic lighting
high intensity light flash
to eliminate motion blur

32

Computer
Vision

Stroboscopic lighting

33