Chapter 9 Image Segmentation

9.0 Introduction
9.1 Detection of Discontinuities
9.1.1 Point Detection
9.1.2 Line Detection
9.1.3 Edge Detection
9.2 Spatial Masks
9.3 GRADIENT BASED EDGE DETECTION
9.3.1 Robert
9.3.2 Sobel
9.3.3 PREWITT
9.4 LAPLACIAN BASED EDGE DETECTION
9.4.1 CANNY'S EDGE DETECTION
9.5 Smoothing
9.6 Edge Linking
9.6.1 Local Processing
9.6.2 Hough Transform
9.7 Thresholding
9.7.1 Global thresholding
9.8 Region based segmentation
9.8.1 What is a region?
9.8.2 Partitioning
9.8.3 Boundaries
9.8.4 Quad-trees
9.8.5 Region growin
9.8.6 Region merging
9.8.7 Split
9.8.8 Split & merge
9.0 Introduction
Image Segmentation is a process of dividing or partitioning the digital image into number of
small segment. The main aim of the Image Segmentation is to simplify and/or change the
representation of Image into suitable format that is easy for understand and analysis.Image
segmentation is used to locate objects and boundaries i..e line ,curve etc of an image.

Image segmentation can used for recognition of an object,image compression,image editing or

image database look up.

The result of image Segmentation consist set of segments that covers the entire image or a set
of contour extracted from an image.In particular region,each pixel are similar having some
characteristics or property such as colour, intensity or texture

9.1 Detection of Discontinuities

In this section we describe several technique for detecting the three basic type of gray level
discontinuities in a digital image i.e points, lines and edges. In order to view the discontinuities
in an images we have to run a mask over the digital image. This procedure involves the sum of
products of the coefficients with the gray level described in the region that encompasses by the
mask

The output of the mask at any point in the image is given by

9
R=w1z1 + w2z2 +……….+ w9z9 = ∑wizi
Where zi is the gray level of the pixel in an image that associated with mask coefficient wi

9.1.1 Point Detection

In point Detection, the main aim is to detect the isolated point in digital image. The important factor is
that it helps us to detect the isolated points or pixel is the difference between their gray levels and gray
levels of their neighborhood pixel . with this observation it suggest that by using mask that
magnify these difference that helps us to distinguished these point from the surrounding pixel.
The mask is designed to amplify the gray level differences of the center pixel from its
neighbours.
The value obtained by the mask is shown as R then based upon these value one can decided
whether the pixel is isolated or not . In Imaging application, the value R Is Compared with a
prespecified threshold T .For any point in the image, the point detection method has the
following condition
R≥T
If the condition is there, then the point is marked as the isolated point that has occur and need
to analysis

9.1.2 Line Detection:

Line detection methods are used a variety of line detection masks that are useful in magnifying
and detecting horizontal lines, vertical or lines with any prespecified angles

The four mask is used for detecting horizontal lines, vertical lines, rising line with the angle 45
degree and falling lines with the angles of -45 degree

if, at a certain point in the image

|Ri| > |Rj|,

for all ji, that point is said to be more likely associated with a line in the direction of mask i.
9.1.3 Edge Detection
Edge detection is used to identifying pixels in the digital image at which image change its
brightness sharply or consist certain discontinuities. The point at which image change its
brightness sharply are typically organized into a set of curved line segments called edges.

The problem is used to find in the 1D signal is called as step detection and the problem used to
find out signal discontinuities overtime is called as change detection. Edge detection is an
important tool in image processing, machine vision and computer vision in the area of feature
detection and feature extraction.

An edge is an boundary between two regions with relative gray level properties. The most edge
detection technique use the concept of relative derivative operator

The following figure illustrates the concepts of relative derivative operator. The picture on the
right shows an image of light stripe on a dark background, the gray level profile of a horizontal
line. And the first and second derivative of the profile. The first Derivative has a positive value
when the image changes from dark to light and zero where the image remains constant. The
second Derivative is positive some part of image transition associated with dark side of the
edge and negative for transition associated with the light side of the edge. The Magnitude of
first derivative is used to detect the presence of edge in the image and the magnitude of the
second derivative is used to detect whether a pixel lies on the dark or light side of an edge.

9.2 Spatial Masks

We used two dimensional spatial mask to find different location in an image. the working
principle of mask is similar to filter except it operates in two dimensionally. The operation of
mask is it centered upon each pixel in an image and calculated the weight
 w_1 w_2 w_3
w_4 w_5 w_6
w_7 w_8 w_9

Mask

9.3 GRADIENT BASED EDGE DETECTION

In the edge detection technique the assumption is made that the edge are with pixel have a
high gradient. The faster change of intensity at some direction given by angle of the gradient
vector is observed at edge pixel. At the pixel level, the intensity of the pixel change from 0 to
255 at the direction of the gradient. The magnitude of the gradient signifies the strength of an
edge . if we calculate the gradient at uniform region we get 0 vector which means that there is
no edge pixel

In natural image we usually do not have the ideal discontinuity or uniform region as in the
figure and we process the magnitude of gradient in order to make decision to detect the edge
pixels. The simplest method is used to applying a threshold. If the gradient magnitude is larger
than the threshold value that will decide the corresponding pixel is an edge pixel.

The edge pixel described the two important features

1) Edge strength, which is equal to the magnitude of the gradient

2) Edge direction , which is equal to the angle of the gradient
9.3.1 Robert
The Robert cross operator provides a simple approximation to the gradient magnitude

Using convolution masks, this becomes

Where Gx and Gy are calculated using the following masks:

As with the previous 2 x 2 gradient operator, the differences are computed at the
interpolated point [i + 1/2, j + 1/2]. The Roberts operator is an approximation to the
continuous gradient at this interpolated point and not at the point [i, j] as might be
expected.

9.3.2 SOBEL
In order to avoid having the gradient calculated about an interpolated point between
pixels is to use a 3 x 3 neighborhood for the gradient calculations. Consider the
arrangement of pixels about the pixel [i, j] shown in the Figure The Sobel operator is the
magnitude of the gradient computed by

where the partial derivatives are computed by:

with the constant c = 2

Like the other gradient operators, Sx and Sy can be implemented using convolution
masks

Note that this operator places an emphasis on pixels that are closer to the center of the
mask. The Sobel operator is one of the most commonly used edge detectors
9.3.3 PREWITT
The Prewitt operator uses the same equations as the Sobel operator, except that the
constant c = 1. Therefore

Note that, unlike the Sobel operator, this operator does not place any emphasis on
pixels that are closer to the center of the masks.
9.4 LAPLACIAN BASED EDGE DETECTION
The edge points of an image can be detected by finding the zero crossings of the second
derivative of the image intensity. The idea is illustrated for a 1D signal in Figure
However, calculating 2nd derivative is very sensitive to noise. This noise should be
filtered out before edge detection. To achieve this, “Laplacian of Gaussian” is used. This
method combines Gaussian filtering with the Laplacian for edge detection.
In Laplacian of Gaussian edge detection there are mainly three steps:

· Filtering,

· Enhancement,

· and detection.

Gaussian filter is used for smoothing and the second derivative of which is used for the
enhancement step. The detection criterion is the presence of a zero crossing in the
second derivative with the corresponding large peak in the first derivative.

In this approach, firstly noise is reduced by convoluting the image with a Gaussian
filter. Isolated noise points and small structures are filtered out. With smoothing;
however; edges are spread. Those pixels, that have locally maximum gradient, are
considered as edges by the edge detector in which zero crossings of the second
derivative are used. To avoid detection of insignificant edges, only the zero crossings
whose corresponding first derivative is above some threshold, are selected as edge
point. The edge direction is obtained using the direction in which zero crossing occurs.

The output of the Laplacian of Gaussian (LoG) operator; h(x,y); is obtained by the
convolution operation:

Where

is commonly called the mexican hat operator.

In the LoG there are two methods which are mathematically equivalent:

· Convolve the image with a gaussian smoothing filter and compute the Laplacian of
the result,

· Convolve the image with the linear filter that is the Laplacian of the Gaussian filter.

This is also the case in the LoG. Smoothing (filtering) is performed with a Gaussian filter,
enhancement is done by transforming edges into zero crossings and detection is done by
detecting the zero crossings.
9.4.1 CANNY'S EDGE DETECTION

Canny Edge Detection Algorithm has the following steps:

· Smooth the image with a Gaussian filter,

· Compute the gradient magnitude and orientation using finite-difference approximations for
the partial derivatives,

· Apply nonmaxima suppression to the gradient magnitude,

· Use the double thresholding algorithm to detect and link edges.

Canny edge detector approximates the operator that optimizes the product of signal-to-noise
9.5 Smoothing:

Let I[I,j] denote the image; be a Gaussian smoothing filter G[I,j, σ] where σ is the spread of
the Gaussian and controls the degree of smoothing. The result of convolution of I[I,j] with G[I,j,
σ] gives an array of smoothed data as:

9.6 Edge Linking

It seem that gradient operators like Roberts, Sobels ,Prewitts which is used enhance the
edges. When these filters applied practically, there are usually creates breaks in lines.
Due to the noise, there are high amount intensity discontinuities because of which
output of a gradient filtered image would have pixels that lie on the edge as well as
pixels that represents noise.
Due to these, the edge detection algorithms are generally followed by edge linking
methods which is used to form edges from edge pixels. This can be done by the help of
joining the edge pixel.

There are two major technique for edge linking

9.6.1 Local Processing

A simplest approach is local processing that used to analysis pixels in a neighborhood
(3X3 or 5X5) of an image that have to undergone to edge detection. In these, all the
points or pixels that share a some common properties are linked together.

There are two common properties that we consider the pixels or points are linked
together
1) Strength of the response of the gradient operator
2) The direction of the gradient

1) The Strength of the Gradient: we already known the gradient operator like
Prewitt and sobel operators.
|F |=[F2x + F2y ] 1/2
Where Fxgradient in the x-direction.
Fygradient in the y- direction.
A pixel in the neighbourhood pixel F(x,y) is linked to the pixel (x,y) if
|F(x,y)- F(x’,y’)|≤T
Where T is a non-negative threshold
F(x,y) means that the image obtained after using the sodel or prewitts filters.
2) The Direction of the Gradient: the direction of the gradient’s vector is given by
the equation
α (x,y)=tan-1 (Fx/Fy)
where Fx x gradient image
Fy y gradient image
A pixel in the neighbourhood pixel α (x,y) is linked to the pixel (x,y) if they have
similar angle is given by following equation
| α (x,y)- α (x’,y’)|< A

Where A is non-negative angle threshold.

Only if both these condition are satisfied , the pixels have come properties are
linked together

9.6.2 Hough Transform

One of the powerful technique for detecting edges is called hough transform
Let us suppose we are looking a straight line in an image.
If we take a point (x’,y’ ) in an image . all lines passes through the point or pixel
have the form
Y’=mx’+c that varying the values m and c.

Fig. Lines through a point

However, this equation can be written as in another way
C=-x’m+y’
where we now consider x' and y' to be constants, and the value of m and c as varying.
This is a straight line on a graph of c against m as shown in Fig.
Each different line through the point (x',y') corresponds to one of the points on the line
in (m,c) space.
 Fig. The C Compilation Model
Now consider two pixels p and q in (x,y) space which lie on the same line.
For each pixel, all of the possible lines through it are represented by a single line
in (m,c) space.
Thus the single line in (x,y) space which goes through both pixels lies on the intersection of the
two lines representing p and q in (m,c) space,is shown in the Fig.

Fig. Points on the same line

Taking this one step further,

 All pixels which lie on the same line in (x,y) space are represented by lines which all pass
through a single point in (m,c) space.
 The single point through which they all pass gives the values of m and c in the equation
of the line y=mx+c.

To detect straight lines in an image, we do:

 Quantise (m,c) space into a two-dimensional array A for appropriate steps of m and c.
 Initialise all elements of A(m,c) to zero.
 For each pixel (x',y') which lies on some edge in the image, we add 1 to all elements of
A(m,c) whose indices m and c satisfy y'=mx'+c.
 Search for elements of A(m,c) which have large values -- Each one found corresponds to
a line in the original image.

One useful property of the Hough transform is that the pixels which lie on the line need not all be
contiguous
For example, all of the pixels lying on the two dotted lines in Fig. will be recognized as lying on
the same straight line. This can be very useful when trying to detect lines with short breaks in
them due to noise, or when objects are partially occluded as shown.

Non-contiguous pixels on the same line

On the other hand, it can also give misleading results when objects happen to be aligned by chance, as
shown by the two dotted lines in Fig. .

Fig. Aligned objects

Indeed, this clearly shows that one disadvantage of the Hough transform method is that it gives an
infinite line as expressed by the pair of m and c values, rather than a finite line segment with two well-
defined endpoints.

One practical detail is that the y=mx+c form for representing a straight line breaks down for
vertical lines, when m becomes infinite.

To avoid this problem, it is better to use the alternative formulation given earlier,

xcosθ + ysinθ==r1

as a means of describing straight lines

9.7 Thresholding

Segmentation divides an image into its constituent regions or objects.Segmentation of non trivial
images is one of the difficult task in image processing. Segmentation accuracy determines the
eventual success or failure of computerized analysis procedure.Example Application: Automated
inspection of electronic assemblies. (mother boards)
Segmentation algorithms are based on one of two basic properties of intensity values
discontinuity and similarity.First category is to partition an image based on abrupt changes in
intensity, such as edges in an image.Second category are based on partitioning an image into
regions that are similar according to a predefined criteria. Histogram Thresholding approach falls
under this category.

Histogram are constructed by splitting the range of the data into equal-sized bins (called
classes). Then for each bin, the number of points from the data set that fall into each bin
are counted. Vertical axis: Frequency (i.e., counts for each bin) Horizontal axis:
Response variable. In image histograms the pixels form the horizontal axis

Thresholding – Foundation

Suppose that the gray-level histogram corresponds to an image, f(x,y), composed of dark
objects in a light background, in such a way that object and background pixels have gray
levels grouped into two dominant modes. One obvious way to extract the objects from
the background is to select a threshold ‘T’ that separates these modes. Then any point
(x,y) for which f(x,y) > T is called an object point, otherwise, the point is called a
background point.

If two dominant modes characterize the image histogram, it is called a bimodal

histogram. Only one threshold is enough for partitioning the image.

If for example an image is composed of two types of light objects on a dark background,
three or more dominant modes characterize the image histogram.

In such a case the histogram has to be partitioned by multiple thresholds.

Multilevel thresholding classifies a point (x,y) as belonging to one object class

if T1 < (x,y) <= T2,

to the other object class

if f(x,y) > T2

and to the background

if f(x,y) <= T1.

Thresholding is the simplest segmentation method. The pixels are partitioned depending
on their intensity value.

Global thresholding, using an appropriate threshold T :

g (x , y ) = 1, if f (x , y ) > T

0, if f (x , y ) ≤ T

Variable thresholding, if T can change over the image.

Local or regional thresholding, if T depends on a neighborhood of (x , y ).

adaptive thresholding, if T is a function of (x , y ).

Multiple thresholding:

a, if f (x , y ) > T2

g (x , y ) = b, if T1 < f (x , y ) ≤ T2

c , if f (x , y ) ≤ T1

Choosing the thresholds

Peaks and valleys of the image histogram can help in choosing the appropriate value for
the threshold(s).

Some factors affects the suitability of the histogram for guiding the choice of the
threshold:

the separation between peaks;

the noise content in the image;

the relative size of objects and background;

 the uniformity of the illumination;

the uniformity of the reflectance.

9.7.1 Global thresholding

A simple algorithm:
1. Initial estimate of T
2. Segmentation using T :
G1, pixels brighter than T ;
G2, pixels darker than (or equal to) T .
3. Computation of the average intensities m1 and m2 of G1
And G2
.4. New threshold value:
Tnew = m1 + m2 /2
5. If |T – Tnew | > ∆T , back to step 2, otherwise stop.

Global thresholding: an example

Local properties based thresholding

Local properties (e.g., statistics) based criteria can be used for adapting the threshold.

For example:

Txy = aσxy + bmxy

Txy = aσxy + bmG

The segmentation is operated using a suitable predicate, Qxy :

g (x , y ) =1, if Qxy

0, otherwise

where Qxy can be, for instance:

f (x , y ) > Txy

f (x , y ) > aσ xy AND f (x , y ) > bmxy This technique can be easily generalized to multiple
thresholdssegmentation.

9.8 Region based segmentation

9.8.1 What is a region?

• Basic definition :- A group of connected pixels with similar properties.

• Important in interpreting an image because they may correspond to objects in a scene.
• For correct interpretation, image must be partitioned into regions that correspond to
objects or parts of an object.

9.8.2 Partitioning

• Partitioning into regions done often by using gray values of the image pixels.
• Two general approaches :-

Region-based segmentation

Boundary estimation using edge detection

Region Segmentation

• Problem Statement:-

Given a set of image pixels I and a homogeneity predicate P(.), find a partition S of the
image I into a set of n regions Ri such that

 Ri  True
i 1

• P(Ri) = True , for all i

 i.e any region satisfies the homogeneity predicate

• Any two adjacent regions cannot be merged into a single region

 
P Ri  Rj  False

Original Image

Region Segmented Image

9.8.3 Boundaries

Extract the boundary and store only the boundary pixels

Advantage:
•Efficient representation of a region
•Storage in chain code
Drawback:
•The region must be reconstructed for further use.
Run-lengthcodes
Any region can be viewed as a sequence of alternating strings of 0s and 1s
Just count the number of 0s, then the number of following 1s and so on…..

9.8.4 Quad-trees

The region is enclosed in a convenient rectangular area

This area is divided into four quadrants until it is totally in fore-or background

• Extension of pyramids for binary images.

• Three types of nodes – white, black, gray.

• White or black node – no splitting.

• Gray node – split into 4 sub-regions.
• Each node is either a leaf or has 4 children.

Quad-trees
Encoding and saving of a quadtree Recursive
•On a leaf: store 0 or 1
•On a node:
-Add a ‘(‘
-store recursively code of the children
-Add a ‘)’

9.8.5 Region growing

Based on seed points:
•Give the initial region
•Set by the user
•Are definitively within the region to be segmented
After kiterations: we have a segmented region
•Look at all neighbours of this region
•Are they homogeneous with respect to the criteria
•Yes: add them
•No: finish region growing
9.8.6 Region merging

Region merging: initial partition of the image into small homogeneous regions (e.g. 2x2 pixels)
•If two neighbours regions are homogeneous then merge these regions
•Continue until no homogeneous neighboured regions are left
No user set seed points is necessary!

9.8.7 Split

The whole image is the initial partition

Test with a criteria if the partition is homogeneous
•If yes: do not split the partition
•Otherwise: split the image, e.g. into 4 rectangles
Continue with all partitions until no further splits are necessary
Possible problem: one region can be split into more than one partition
Advantages:

•No seed-points
•Fast and easy
Drawback: over-segmentation
•Usually too many regions are created

9.8.8 Split & merge

Extension of split
First step: do a split!
Then: merge homogeneous partitions
•Check all pairs of neighboured partition if they are
homogeneous with the same criteria
•If yes: merge then
Finish when all partitions are homogeneous