ENEE631 Spring09 Lecture-1 (1/26/2009)

Digital Image and Video Processing An Introduction

Spring 09 Instructor: Min Wu
Electrical and Computer Engineering Department University of Maryland, College Park
bb.eng.umd.edu (select ENEE631 S09) minwu@eng.umd.edu
M. Wu: ENEE631 Digital Image Processing (Spring'09)

ENEE631 Logistics Spring 2009

Lectures
Monday and Wednesday 11am-12:15pm, CSI 2120

Assignments and Projects

Matlab will be used for many assignments; C/C++ may also be involved in some. Kim Lab #2107 ~ image/video related software installed for EE408G

students are encouraged to make use of them in public lab hours.

Office Hours
Dr. Min Wu (minwu@eng.umd.edu)

Wednesday 12:30 2:30pm @ Kim 2142, or by appointment

Regularly check the course web page

bb.eng.umd.edu

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [2]

Scope of ENEE631

First graduate course on image/video processing Prerequisites: ENEE620 and 630, or by permission
Not assume you have much exposure on image processing at undergraduate level Require and build on background in random process and DSP

Emphasis on fundamental concepts

Provide theoretical foundations on multi-dimensional signal processing built upon pre-requisites Coupled with assignments and projects for hands-on experience and reinforcement of the concepts Follow-up courses

image analysis, computer vision, pattern recognition multimedia communications and security
Lec1 Introduction [3]

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Textbooks and References

R.C. Gonzalez and R.E. Woods: Digital Image Processing, Prentice Hall, 3rd Edition, 2008. (yellow cover)

Related technical publications (will be announced in class) Other related textbooks

Y. Wang, J. Ostermann, Y-Q. Zhang: Digital Video Processing and Communications, Prentice Hall, 2001.
A.K. Jain: Fundamentals of Digital Image Processing, Prentice Hall, 1989. John W. Woods: Multidimensional Signal, Image, and Video Processing and Coding, Academic Press, 2006. A.Bovik: Handbook Of Image & Video Processing, 2nd Edition, Academic Press, 2005.
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [4]

ENEE631 Course Organization

Grading
Assignments and class participation Projects Exams 20% 45% 35%

Assignments: theoretical problems + computer components

Involves Matlab or C/C++ programming and tasks with image/video tools to reinforce concepts Grading is based mainly on completeness; encourage further explorations and discussions

Projects
Put theories and principles in use and learn from doing; critical thinking

Exams
In-class mid-term exam: on basic concepts, theories, and approaches Final exam: apply theories and principles to image/video proc tasks
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [5]

ENEE631 Course Policies

No late submission will be accepted

Start early! Plan wisely and prepare for unforeseen hurdles Inform instructor of special circumstances with documentation

Independent work vs. discussions

Write up your solutions INDIVIDUALLY Discussions with classmates on assignments and projects are encouraged (unless otherwise noted)

Computer codes
You should write your own codes unless otherwise stated DO NOT COPY other students codes Clearly state the code modules obtained elsewhere and consult instructor for permission to use in your project

Academic integrity: cheating, plagiarism, fabrication of results,

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [7]

UMCP ENEE631 Slides (created by M.Wu 2001)

Image and Video Processing: An Introduction and Overview

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [8]

A picture is worth 1000 words.

UMCP ENEE631 Slides (created by M.Wu 2001)

A video is worth 1000 sentences?

http://marsrovers.jpl.nasa.gov/gallery/press/opportunity/20040125a.html JPL Mars Panorama captured by the Opportunity

Rich info. from visual data Examples of images around us

natural photographic images; artistic and engineering drawings scientific images (satellite, medical, etc.)

Motion pictures => video

movie, TV program; family video; surveillance and highway/ferry camera
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [9]

Increasing Use of Images A Glimpse from Encyclopedia Britannica

(From B. Liu EE488 F06 at Princeton)

First Edition (1768-1771) A dictionary of arts and sciences by a Society of Gentlemen in Scotland 3 volumes, ~ 2600 pages illustrated with 160 copperplates 11th Edition (1911) last time to encapsulate ALL human knowledge one picture every 4 pages 1999 Edition 32 volumes; in CD and DVD 73,000 articles; 30,000 photos and illustrations Now online: http://www.britannica.com/
Lec1 Introduction [10]

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Why Do We Process Images?

UMCP ENEE631 Slides (created by M.Wu 2001)

Enhancement and restoration

Remove artifacts and scratches from an old photo/movie Improve contrast and correct blurred images

Composition (for magazines and movies), Display, Printing Transmission and storage
images from oversea via Internet, or from a remote planet

Information analysis and automated recognition

Providing human vision to machines

Medical imaging for diagnosis and exploration

Security, forensics and rights protection
Encryption, hashing, digital watermarking, digital fingerprinting
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [11]

Why Digital?
UMCP ENEE631 Slides (created by M.Wu 2001)

Exactness
Perfect reproduction without degradation Perfect duplication of processing result

Convenient & powerful computer-aided processing

Can perform sophisticated processing through computer hardware or software Even kindergartners can do some!

Easy storage and transmission

1 CD can store hundreds of family photos! Paperless transmission of high quality photos through network within seconds

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [12]

Examples of Digital Image & Video Processing

UMCP ENEE631 Slides (created by M.Wu 2001)

Compression Manipulation and Restoration

Restoration of blurred and damaged images Noise removal and reduction Morphing

Applications
Visual mosaicing and virtual views Face detection Visible and invisible watermarking Error concealment and resilience in video transmission

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [13]

Compression

UMCP ENEE631 Slides (created by M.Wu 2001)

Color image of 600x800 pixels

Without compression

600*800 * 24 bits/pixel = 11.52K bits = 1.44M bytes

After JPEG compression (popularly used on web)

only 89K bytes compression ratio ~ 16:1

Movie ~ Image Sequence

720x480 per frame, 30 frames/sec, 24 bits/pixel Raw video ~ 243M bits/sec DVD ~ about 5M bits/sec Compression ratio ~ 48:1
M. Wu: ENEE631 Digital Image Processing (Spring'09)

Library of Congress by M.Wu (600x800)

Lec1 Introduction [14]

Denoising
UMCP ENEE631 Slides (created by M.Wu 2001)

From X.Li http://www.ee.princeton.edu/~lixin/denoising.htm

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [15]

Deblurring
UMCP ENEE631 Slides (created by M.Wu 2001)

http://www.mathworks.com/access/helpdesk/help/toolbox/images/deblurr7.shtml

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [16]

Special Effects: Morphing

UMCP ENEE631 Slides (created by M.Wu 2001)

Princeton CS426 face morphing examples http://www.cs.princeton.edu/courses/archive/fall98/cs426/assignments/morph/morph_results.html

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [17]

Visual Mosaicing
Stitch photos together without thread or scotch tape
UMCP ENEE631 Slides (created by M.Wu 2001)

R. Radke Princeton thesis 5/2001

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [18]

Face Detection
UMCP ENEE631 Slides (created by M.Wu 2001)

Face detection in 98 @ CMU CS, http://www.cs.cmu.edu/afs/cs/Web/People/har/faces.html

Image enhancement, feature extractions, and statistical modeling are often important steps in computer vision tasks

See more image understanding examples by Prof. Chellappas research group (http://www.cfar.umd.edu/~rama/research.html)

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [19]

General Illumination Correction and Its Application to Face Normalization, J. Zhu et al, ICASSP 2003
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [20]

Visible Digital Watermarks

UMCP ENEE631 Slides (created by M.Wu 2001)

from IBM Watson web page Vatican Digital Library

Lec1 Introduction [22]

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Invisible Watermark
UMCP ENEE631 Slides (created by M.Wu 2001; 2009)

Original, marked, and their amplified luminance difference human visual model for imperceptibility: protect smooth areas and sharp edges
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [23]

Data Hiding for Annotating Binary Line Drawings

UMCP ENEE631 Slides (created by M.Wu 2001)

original

marked w/ 01/01/2000

pixel-wise difference

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [25]

Error Concealment
(a) original lenna image
UMCP ENEE631 Slides (created by M.Wu 2001)

(b) corrupted lenna image

(c) concealed lenna image

25% blocks in a checkerboard pattern are corrupted

corrupted blocks are concealed via edge-directed interpolation

Examples were generated using the source codes provided by W.Zeng.

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [26]

2009 International Conf. on Image Processing (ICIP)

According to the Call-for-Paper

(Cairo, Egypt, Nov. 2009)

http://icip2009.org/

16th in the series (since 1994)

Research frontiers ranging from traditional image processing applications to evolving multimedia and video technologies Areas of interest include but are not limited to: Image/Video Coding and Transmission: Still image coding, video coding,
stereoscopic and 3-D coding, distributed source coding, . . . Image/Video Processing: filtering, restoration, enhancement, segmentation, video segmentation and tracking, morphological processing, stereoscopic and 3-D processing, feature extraction and analysis, interpolation and super-resolution, motion detection and estimation, . . . Image Formation: Biomedical imaging, remote sensing, geophysical and seismic imaging, optimal imaging, synthetic-natural hybrid image systems Image Scanning, Display, and Printing: Scanning, sampling, quantization and halftoning, color reproduction, image representation and rendering, Image/Video Storage, Retrieval, and Authentication: Image/video databases, image/video indexing and retrieval, multimodality image/video indexing and retrieval, authentication and watermarking Applications: biomedical sciences, geosciences and remote sensing, . . .
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [27]

UMCP ENEE408G Slides (created by M.Wu & R.Liu 2002)

So Whats a Digital Image After All?

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [28]

What is an Image?

UMCP ENEE631 Slides (created by M.Wu 2007)

What we perceive as a grayscale image is a pattern of light intensity over a 2-D plane (aka image plane)
Described by a nonnegative real-valued function I(x,y) of two continuous spatial coordinates on an image plane. I(x,y) is the intensity of the image at the point (x,y). An image is usually defined on a bounded rectangle for processing
I: [0, a] [0, b] [0, inf )
x

Color image
Can be represented by three functions:
R(x,y) for red, G(x,y) for green, B(x,y) for blue.

y
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [30]

Different Ways to View an Image

(More generally, to view a 2-D realvalued function)

Intensity visualization over 2-D (x,y) plane

In 3-D (x,y, z) plot with z=I(x,y); red color for high value and blue for low

Equal value contour in (x,y) plane

Example from B. Liu EE488 F06 Princeton

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [31]

Sampling and Quantization

UMCP ENEE631 Slides (created by M.Wu 2001)

Computer handles discrete data. Sampling

Sample the value of the image at the nodes of a regular grid on the image plane. A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j).

Quantization
Is a process of transforming a real valued sampled image to one taking only a finite number of distinct values. Each sampled value in a 256-level grayscale image is represented by 8 bits.
=> Stay tuned for the theories on these in future weeks.
M. Wu: ENEE631 Digital Image Processing (Spring'09)

255 (white)

0 (black)

Lec1 Introduction [32]

Recall: 1-D Sampling Theorem

UMCP ENEE631 Slides (created by M.Wu 2002)

1-D Sampling Theorem

A 1-D signal x(t) bandlimited within [-B,B] can be uniquely determined by its samples x(nT) if s > 2B (i.e. sample fast enough). Using the samples x(nT), we can reconstruct x(t) by filtering the impulse version of x(nT) by an ideal low pass filter

Sampling below Nyquist rate (2B) cause Aliasing

Xs() with s > 2B Perfect Reconstructable Xs() with s < 2B Aliasing

-s

B s=2/T

B s=2/T

=> Will extend sampling theorem to 2-D later in the course

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [33]

Examples of Sampling
UMCP ENEE631 Slides (created by M.Wu 2001)

256x256 64x64 16x16

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [36]

Examples of Quantizaion
UMCP ENEE631 Slides (created by M.Wu 2001)

8 bits / pixel 4 bits / pixel 2 bits / pixel

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [37]

An Ancient Example of Digital Image

An Old Digital Picture

(from a small church in Crete Island, Greece)

=> Colored tiles as pixels

Slide from B. Liu EE488 F06 Princeton

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [38]

Summary of Todays Lecture

UMCP ENEE631 Slides (created by M.Wu 2001)

Course organization and policies Background and examples of digital image processing Sampling and quantization concepts for digital image

Next time
Color and Human Visual System

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [39]

Readings and Assignment

UMCP ENEE631 Slides (created by M.Wu 2001,2004,2009)

Introductory sections in Matlab Image Processing Toolbox

http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml

Gonzalez-Wood book, Chapter 1 Boviks Handbook Section 1 Introduction (see course web) Go over mathematical preliminaries
Linear system and basics of 1-D signal processing FT and ZT Matrix and linear algebra Probability

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [40]

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [41]

Color of Light
UMCP ENEE408G Slides (created by M.Wu & R.Liu 2002)

Perceived color depends on spectral content

(wavelength composition)
e.g., 700nm ~ red. spectral color

A light with very narrow bandwidth

Spectrum from http://www.physics.sfasu.edu/astro/color.html

A light with equal energy in all visible bands appears white

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 Introduction [42]

Perceptual Attributes of Color

UMCP ENEE408G Slides (created by M.Wu & R.Liu 2002)

Value of Brightness (perceived luminance)

Chrominance Hue

specify color tone (redness, greenness,

etc.)

depend on peak wavelength

Saturation

describe how pure the color is depend on the spread (bandwidth) of light spectrum reflect how much white light is added

HSV circular cone is from online documentation of Matlab image processing toolbox http://www.mathworks.com/access /helpdesk/help/toolbox/images/col or10.shtml
Lec1 Introduction [43]

RGB HSV Conversion ~ nonlinear

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Questions for Today (QFT)

Seeing yellow figure is from B.Liu ELE330 S01 lecture notes @ Princeton; primary color figure is from Chapter 6 slides at Gonzalez/ Woods DIP book website

Why seeing yellow without yellow?

mix green and red light to obtain the perception of yellow, without shining a single yellow light
570nm 520nm 630nm

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Lec1 Introduction [44]