Project Report New 03

1
INTRODUCTION
Science is wonderfully equipped to answer the question "How?" but it gets terribly confused when you ask the question "Why?" - Erwin Chargaff
CHAPTER 1 INTRODUCTION
SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010
As processors become increasingly powerful, and memories become increasingly cheaper, the deployment of large image databases for a variety of applications have now become realizable. Databases of art works, satellite and medical imagery have been attracting more and more users in various professional fields for example, geography, medicine, architecture, advertising, design, fashion, and publishing. Effectively and efficiently accessing desired images from large and varied image databases is now a necessity. 1.1 DEFINITION CBIR or Content Based Image Retrieval is the retrieval of images based on visual features such as colour, texture and shape [17]. Reasons for its development are that in many large image databases, traditional methods of image indexing have proven to be insufficient, laborious, and extremely time consuming. These old methods of image indexing, ranging from storing an image in the database and associating it with a keyword or number, to associating it with a categorized description, have become obsolete . This is not CBIR. In CBIR, each image that is stored in the database has its features extracted and compared to the features of the query image. It involves two steps [17]:

Feature Extraction: The first step in the process is extracting image features to a distinguishable extent. Matching: The second step involves matching these features to yield a result that is visually similar.
1.1.1 Applications of CBIR Examples of CBIR applications are [17]:
Crime prevention: Automatic face recognition systems, used by police forces.

2
Security Check: Finger print or retina scanning for access privileges. Medical Diagnosis: Using CBIR in a medical database of medical images to aid diagnosis by identifying similar past cases. Intellectual Property: Trademark image registration, where a new candidate mark is compared with existing marks to ensure no risk of confusing property ownership.
1.1.2 CBIR Systems Several CBIR systems currently exist, and are being constantly developed. Few Examples are:
QBIC or Query By Image Content was developed by IBM, Almaden Research Centre, to allow users to graphically pose and refine queries based on multiple visual properties such as colour, texture and shape [16]. It supports queries based on input images, user-constructed sketches, and selected colour and texture patterns [16].
VIR Image Engine by Virage Inc., like QBIC, enables image retrieval based on primitive attributes such as colour, texture and structure. It examines the pixels in the image and performs an analysis process, deriving image characterization features [16].
VisualSEEK and WebSEEK were developed by the Department of Electrical Engineering, Columbia University. Both these systems support colour and spatial location matching as well as texture matching [16].
NeTra was developed by the Department of Electrical and Computer Engineering, University of California. It supports colour, shape, spatial layout and texture matching, as well as image segmentation [16].
MARS or Multimedia Analysis and Retrieval System was developed by the Beckman Institute for Advanced Science and Technology, University of Illinois. It supports colour, spatial layout, texture and shape matching [16].
Viper or Visual Information Processing for Enhanced Retrieval was developed at the Computer Vision Group, University of Geneva. It supports colour and texture matching [16].
1.2 PROBLEM MOTIVATION
Present image search systems(search engines) are mostly text based retrieval systems. Many times this leads to unwanted results. Current techniques are based on low level features like colour, texture,shape etc. and there is a huge semantic gap existing. Image databases and collections can be enormous in size, containing hundreds, thousands or even millions of images. The conventional method of image retrieval is searching for a keyword that would match the descriptive keyword assigned to the image by a human categorizer . Currently under development, even though several systems exist, is the retrieval of images based on their content, called Content Based Image Retrieval, CBIR. While computationally expensive, the results are far more accurate than conventional image indexing. Hence, there exists a tradeoff between accuracy and computational cost. This tradeoff decreases as more efficient algorithms are utilized and increased computational power becomes inexpensive.
1.3 PROBLEM STATEMENT The problem involves entering an image as a query into a software application that is designed to employ CBIR techniques in extracting visual properties, and matching them. This is done to retrieve images in the database that are visually similar to the query image.
2
the world
LITERATURE
SURVEY
A mans feet should be planted in his country, but his eyes should survey -George Santayana
CHAPTER 2 LITERATURE SURVEY

There are many features of an image that can be extracted from the image itself e.g. colour,texture,shape etc. Some of the features can be explained as follows:
2.1 Colour
2.1.1 Definition One of the most important features that make possible the recognition of images by humans is colour. Colour is a property that depends on the reflection of light to the eye and the processing of that information in the brain[15]. We use colour everyday to tell the difference between objects, places, and the time of day[18] . Usually colours are defined in three dimensional colour spaces. These could either be RGB (Red, Green, and Blue), HSV (Hue, Saturation, and Value) or HSB (Hue, Saturation, and Brightness). The last two are dependent on the human perception of hue, saturation, and brightness[18]. Most image formats such as JPEG, BMP, GIF, use the RGB colour space to store information [18]. The RGB colour space is defined as a unit cube with red, green, and blue axes. Thus, a vector with three co-ordinates represents the colour in this space. When all three coordinates are set to zero the colour perceived is black. When all three coordinates are set to 1 the colour perceived is white [18]. The other colour spaces operate in a similar fashion but with a different perception. 2.1.2 Methods of Representation The main method of representing colour information of images in CBIR systems is through colour histograms. A colour histogram is a type of bar graph, where each bar represents a particular colour of the colour space being used. In MATLAB for example you can get a colour histogram of an image in the RGB or HSV colour space. The bars in a colour histogram are referred to as bins and they represent the x-axis [14][13]. The number of bins depends on the number of colours there are in an image. The y-axis denotes the number of pixels there are in each bin. In other words how many pixels in an image are of a particular colour [14].
An example of a colour histogram in the HSV colour space can be seen with the following image:
Figure 2.1(a): Sample Image (Source: Internet)
Figure 2.1(b): H,S,V components of sample image
As shown in figure we have plotted histograms and image planes corresponding to H,S,V components of sample image. Quantization in terms of colour histograms refers to the process of reducing the number of bins by taking colours that are very similar to each other and putting them in the same bin. By default the number of bins one can obtain using the histogram function in MATLAB is 256. HSV colour space is used for extracting colour features because of following possible advantages: HSV colour space is designed to be similar to human colour perception. In HSV colour space, H and S planes are more important because they represent colour; V represents value or intensity of grayscale image. Hence we can give less importance to V plane of the image. This results in greater computational efficiency i.e. improved time complexity.
2.2 Texture
2.2.1 Definition Texture is that innate property of all surfaces that describes visual patterns, each having properties of homogeneity. It contains important information about the structural arrangement of the surface, such as; clouds, leaves, bricks, fabric, etc. It also describes the relationship of the surface to the surrounding environment [1]. In short, it is a feature that describes the distinctive physical composition of a surface.
Texture properties include: Coarseness Contrast Directionality Line-likeness Regularity Roughness (a) Clouds (c) Rocks
(b) Bricks Figure 2.2: Examples of Textures. (Source: Internet)
Texture is one of the most important defining features of an image. It is characterized by the spatial distribution of gray levels in a neighborhood [1][6]. In order to capture the spatial dependence of gray-level values, which contribute to the perception of texture, a two-dimensional dependence texture analysis matrix is taken into consideration. This two-dimensional matrix is obtained by decoding the image file; jpeg, bmp, etc.
10
2.2.2 Methods of Representation There are three principal approaches used to describe texture; statistical, structural and spectral Statistical techniques characterize textures using the statistical properties of the grey levels of the points/pixels comprising a surface image. Typically, these properties are computed using: the grey level co-occurrence matrix of the surface, or the wavelet transformation of the surface. Structural techniques characterize textures as being composed of simple primitive structures called texels (or texture elements). These are arranged regularly on a surface according to some surface arrangement rules.
Spectral techniques are based on properties of the Fourier spectrum and describe global periodicity of the grey levels of a surface by identifying highenergy peaks in the Fourier spectrum [7].
For optimum classification purposes, what concern us are the statistical techniques of characterization. This is because it is these techniques that result in computing texture properties. The most popular statistical representations of texture are: Co-occurrence Matrix Wavelet Packet Decomposition
11
2.2.2.1 Co-occurrence Matrix Originally proposed by R.M. Haralick, the co-occurrence matrix representation of texture features explores the grey level spatial dependence of texture. A mathematical definition of the co-occurrence matrix is as follows [12]:
-
Given a position operator P(i,j), let A be an n x n matrix whose element A[i][j] is the number of times that points with grey level (intensity) g[i] occur, in the position specified by P, relative to points with grey level g[j].
Let C be the n x n matrix that is produced by dividing A with the total number of point pairs that satisfy P. C[i][j] is a measure of the joint probability that a pair of points satisfying P will have values g[i], g[j].
C is called a co-occurrence matrix defined by P.
Examples for the operator P are: i above j, or i one position to the right and two below j, etc. [12] This can also be illustrated as follows Let t be a translation, then a co-occurrence matrix Ct of a region is defined for every grey-level (a, b) by [3] :
C t ( a , b ) = c a {r sd(, s + t ) R 2 | A [ s ] = a , A [ s + t ] = b }
..(2.1)
Here, Ct(a, b) is the number of site-couples, denoted by (s, s + t) that are separated by a translation vector t, with a being the grey-level of s, and b being the grey-level of s + t. For example; with an 8 grey-level image representation and a vector t that considers only one neighbour, we would find [3]:
12
1 2 2 3
2 3 3 3
1 1 1 2
3 2 2 1
4 4 4 1
Figure 2.3: Image example (8 grey-level image)
0 0 1 2 3 4 5 6 7 0 0 0 0 0 0 0 0
1 0 1 1 0 0 0 0 0
2 0 2 0 1 0 0 0 0
3 0 0 2 1 0 0 0 0
4 0 0 0 0 1 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
Figure 2.4 : Classical Co-occurrence matrix At first the co-occurrence matrix is constructed, based on the orientation and distance between image pixels. Then meaningful statistics are extracted from the matrix as the texture representation [1]. Haralick proposed the following texture features [8]: 1. Angular Second Moment 2. Contrast 3. Correlation 4. Variance 5. Inverse Second Differential Moment 6. Sum Average 7. Sum Variance
13
8. Sum Entropy 9. Entropy 10. Difference Variance 11. Difference Entropy 12. Measure of Correlation 1 13. Measure of Correlation 2 14. Local Mean Hence, for each Haralick texture feature, we obtain a co-occurrence matrix. These co-occurrence matrices represent the spatial distribution and the dependence of the grey levels within a local area. Each (i,j)
th
entry in the matrices, represents the probability of
going from one pixel with a grey level of 'i' to another with a grey level of 'j' under a predefined distance and angle. From these matrices, sets of statistical measures are computed, called feature vectors [1].
14
2.2.2.2 Wavelet Packet Decomposition Textures can be modeled as quasi-periodic patterns with spatial/frequency representation. The wavelet transform transforms the image into a multi-scale representation with both spatial and frequency characteristics. This allows for effective multi-scale image analysis with lower computational cost [4]. According to this transformation, a function, which can represent an image, a curve, signal etc., can be described in terms of a coarse level description in addition to others with details that range from broad to narrow scales [4]. Unlike the usage of sine functions to represent signals in Fourier transforms, in wavelet transform, we use functions known as wavelets. Wavelets are finite in time, yet the average value of a wavelet is zero [4]. In a sense, a wavelet is a waveform that is bounded in both frequency and duration. While the Fourier transform converts a signal into a continuous series of sine waves, each of which is of constant frequency and amplitude and of infinite duration, most real-world signals (such as music or images) have a finite duration and abrupt changes in frequency. This accounts for the efficiency of wavelet transforms. This is because wavelet transforms convert a signal into a series of wavelets, which can be stored more efficiently due to finite time, and can be constructed with rough edges, thereby better approximating real-world signals [9].
Mathematically, we denote a wavelet as [4]
a, b(t) =
(2.2)
Where, b is location parameter and a is scaling parameter. For function to be a Wavelet, it should be time limited We define Wavelet transform as [4]
W(a, b) =
dt
.(2.3)
15
Examples of wavelets are Coiflet, Morlet, Mexican Hat, Haar and Daubechies. Of these, Haar is the simplest and most widely used, while Daubechies have fractal structures and are vital for current wavelet applications [4]. These two are outlined below:
Haar Wavelet
The Haar wavelet family is defined as [4]:
-1
Figure 2.5 : Haar Wavelet Example [4]
Haar wavelet scaling function is defined as:
1 (t) = -1 ..(2.4) 0
0 <= t <= <=t<=1 otherwise
16
Daubechies Wavelet
The Daubechies wavelet family is defined as:
Figure 2.6 : Daubechies Wavelet Example (Source: Internet)
Daubechies wavelet scaling function is defined as:
(2.5)
Where, Ck=h(k).
Ck is unnormalized coefficient and h(k) is normalized coefficient.
17
2.3 SHAPE
2.3.1 Definition Shape may be defined as the characteristic surface configuration of an object; an outline or contour. It permits an object to be distinguished from its surroundings by its outline [10]. Shape representations can be generally divided into two categories [11]: Boundary-based, and Region-based.
Figure 2.7: Boundary-based & Region-based Shape representations(Source: Internet)
Boundary-based shape representation only uses the outer boundary of the shape. This is done by describing the considered region using its external characteristics; i.e., the pixels along the object boundary. Region-based shape representation uses the entire shape region by describing the considered region using its internal characteristics; i.e., the pixels contained in that region [11].
18
2.3.2 Methods of Representation For representing shape features mathematically, we have [11]: Boundary-based: Polygonal Models, boundary partitioning Fourier Descriptors Splines, higher order constructs Curvature Models Superquadrics Fourier Descriptors Implicit Polynomials Blum's skeletons
Region-based:
The most successful representations for shape categories are Fourier Descriptors and Moment Invariants [11]: The main idea of Fourier Descriptor is to use the Fourier transformed boundary as the shape feature. The main idea of Moment invariants is to use region-based moments, which are invariant to transformations as the shape feature.
19
CBIR METHODOLOGY
The whole is equal to the sum of its parts -Euclid The whole is greater than the sum of its parts -Max Wertheimer
20
CHAPTER 3 CBIR METHODOLOGY

The solution proposed is to extract the primitive features of a query image and compare them to those of database images. The image features under consideration are colour, texture and shape. Thus, using matching and comparison algorithms, the colour, texture and shape features of one image are compared and matched to the corresponding features of another image. This comparison is performed using colour, texture and shape distance metrics. In the end, these metrics are performed one after another, so as to retrieve database images that are similar to the query. The similarity between features is to be calculated using specific algorithm. For each specific feature there is a specific algorithm for extraction and another for matching. Relevance Feedback USE Query Formation
Visual Content Description
Feature Vectors Similarity Comparison
Image Database
Visual Content Description
Feature Database Indexing & Retrieval
OUTPUT
Retrieval results
Figure 3.1: CBIR Methodology
21
4
things that wont work.
IMPLEMENTATION & RESULTS
Results! Why, man, I have got a lot of results. I know several thousand -Thomas A. Edison
22
CHAPTER 4 IMPLEMENTATION & RESULTS 4.1 COLOUR

Retrieving images based on colour similarity is achieved by computing a colour histogram for each image that identifies the proportion of pixels within an image holding specific values (that humans express as colours). Examining images based on the colours they contain is one of the most widely used techniques because it does not depend on image size or orientation.
Algorithm Colour Histogram Matching : Feature Matching Most commercial CBIR systems include colour histogram as one of the features (e.g., QBIC of IBM).
(4.1)
Where, j denotes the various gray levels Hist_query is the histogram of query image, Hist_database is histogram of the database image. Histdist is error difference or distance metric The nearest matching database images with the query image has the least distance metric. The exact match is the one with the zero distance metric.
23
Separate H, S, V Planes Precalculate & save histogram each plane (H,S,V) in advance
Compute the histogram for each plane
Database
Compare histogram
Show retrieved images
Figure 4.1 : Flow of implemented image retrieval system
In the figure above, once a query image and a retrieval method are chosen by users, the rest of whose process is done automatically. However, the histogram data for all images in database are computed and saved in Database in advance so that only the image indexes and histogram data can be used to compare the query image with images in Database.
4.1.1 RESULTS (colour)

24
Here is one of the results using colour histogram matching. QUERY IMAGE:-
Figure 4.2(a):A Query Image(1) (Source: Internet)
RETRIEVED IMAGES:
Figure 4.2(b): Retrieved Images for query image(1)
25
QUERY IMAGE:-
RETRIEVED IMAGES :-

26
DISCUSSION:
Colour based image retrieval gives intended results for images containing natural beauty and similar kind of images. It doesnt produce intended search results for images having specific shape or structure. We should go for texture as well as shape features of an image to overcome the shortcomings of colour based retrieval.
27
4.2 TEXTURE
There are three principal approaches used to describe texture: Statistical, Structural, Spectral. For optimum classification purposes, what concern us are the statistical techniques of characterization. This is because it is these techniques that result in computing texture properties. The most popular statistical representations of texture are:
Co-occurrence Matrix Wavelet Packet Decomposition
4.2.1 Co-occurrence Matrix Co-occurrence matrices are a popular representation for the texture in image. They contain a count of the number of times that a given feature (e.g. given gray level) occurs in a particular spatial relation to another given feature. However, because of the large number of spatial relations that are possible within an image, heuristic or interactive techniques have usually been employed to select the relation to use for each problem. In this project we present a statistical approach for finding those spatial relations that best capture the structure of textures when the co-occurrence matrix representation is used. These matrices should thus be well suited for discriminations that are structurally based[7].
28
We have used graycomatrix ( ) function in MATLAB for finding gray-level cooccurrence matrix from the image. glcms = graycomatrix(I,param1,val1,param2,val2,...) It returns one or more gray-level co-occurrence matrices, depending on the values of the optional parameter/value pairs. One of the parameters that can be passed to graycomatrix function is offset. Because the offset is often expressed as an angle, the following table lists the offset values that specify common angles, given the pixel distance D. Angle 0 45 90 135 Offset [0 D] [-D D] [-D 0] [-D -D]
Table 4.1: Offset values for corresponding angles
The figure 4.4 illustrates the array: offset = [0 1; -1 1; -1 0; -1 -1]. 900 [-1 0] 1350 [-1 -1] 00 [0 1] Figure 4.4: Representation of array offset (Source: MATLAB Help) We have considered different directions for calculating distance between pixel of interest and its neighbour. We found that 900 having corresponding offset of [-1 0] yields better retrieval results.
450 [-1 1]
Pixel-of-interest
29
After calculating co-occurrence matrix representation for a gray scale image we found standard deviation for each column. We have compared this standard deviation vector with the extracted standard deviation vectors of corresponding co-occurrence matrices of images in database. Based on the similarity between these standard deviation vectors similar images are retrieved.
(4.2)
Where,
j= number of image in database std_coocc_query = standard deviation vector of co-occurrence matrix for corresponding query image, std_coocc _database = standard deviation vector of co-occurrence matrix of corresponding database image. coocc_dist = error difference or distance metric. The nearest matching database images with the query image has the least distance metric. The exact match is the one with the zero distance metric.
Calculate co-occurrence matrix of query image Precalculate & save standard deviation vectors in advance
Determine standard deviation for each column of co-occ matrix
Database Features
Compare standard deviation vectors
Show retrieved images
Figure 4.5 : Flow of co-occurrence matrix based image retrieval system

30
In the figure above, once a query image is chosen by users, the rest of process is done automatically. However, the co-occurrence matrix data for all images in database are computed and saved in Database in advance so that only the image indexes and cooccurrence data can be used to compare the query image with images in Database.
31
4.2.1.1 RESULTS (co-occurrence)

Here is one of the results using co-occurrence matrix matching algorithm.
QUERY IMAGE:-
RETRIEVED IMAGES :-

32
QUERY IMAGE:-
Figure 4.7(a):A Query Image(4)
RETRIEVED IMAGES :-
33
4.2.2 Wavelet Packet Decomposition In wavelet analysis, a signal is split into an approximation and a detail. The approximation is then itself split into a second-level approximation and detail, and the process is repeated. For an n-level decomposition, there are n+1 possible ways to decompose or encode the signal.
S A1 A2 A3
D3
D1 S2
S = A1 + D1 = A2 + S2 + D1 = A3 + D3 + S2 + D1
Figure 4.8 : Wavelet packet Decomposition
In wavelet packet analysis, the details as well as the approximations can be split. This yields more than packet decomposition tree. different ways to encode the signal. This is the wavelet
S A1 AA2 DA2 AD2 D1 DD2
AAA3
DAA3
ADA3
DDA3
AAD3
DAD3
ADD3
DDD3
Figure 4.9 : 2-D Wavelet packet Decomposition
34
The wavelet packet method is a generalization of wavelet decomposition that offers a richer range of possibilities for signal analysis. The wavelet decomposition tree is a part of this complete binary tree. In the wavelet packet decomposition, each detail coefficient vector is also decomposed into two parts using the same approach as in approximation vector splitting. This offers the richest analysis: the complete binary tree is produced in the onedimensional case or a quaternary tree in the two-dimensional case. Two-dimensional DWT leads to a decomposition of approximation coefficients at level j in four components: the approximation at level j+1, and the details in three orientations (horizontal, vertical, and diagonal). We have used wpdec2 function which is a two-dimensional wavelet packet analysis function. T = wpdec2(X,N,'wname') returns a wavelet packet tree T corresponding to the wavelet packet decomposition of the matrix X, at level N, with a particular wavelet. wname is the name of wavelet filter used. We have used db4 filter from Daubechies wavelet filter family. We have found empirically that db4 is optimum filter for our application in terms of efficiency of retrieval and time complexity.
35
Algorithm : Feature Matching using wavelet packet decomposition 1. Decompose the image using 2-D wavelet packet decomposition 2. Calculate energy corresponding to the terminal nodes of the tree T using wenergy(T) function. 3. Select energy of 5th to 16th terminal nodes of tree . 4. Calculate Euclidian distance for these nodes.
(4.3)
Where, n denotes the no. of images in database. wpd_query is the W.P.D. energy vector of query image, wpd_database is W.P.D. energy vector of the database image. wpd_dist is error difference or distance metric W.P.D. =Wavelet Packet Decomposition The nearest matching database images with the query image has the least distance metric. The exact match is the one with the zero distance metric.
36
4.2.2.1 RESULTS (wavelet packet decomposition)

Here is one of the results using co-occurrence matrix matching algorithm. QUERY IMAGE:-
RETRIEVED IMAGES :-
37
QUERY IMAGE:-
RETRIEVED IMAGES :-
38
DISCUSSION:
Co-occurrence matrix based image retrieval yields good results for images containing certain structure. Wavelet packet decomposition based image retrieval yields even more intended results for images having specific structure.
This was the drawback of colour based image retrieval which is removed by wavelet decomposition.
39
4.3 SHAPE
Shape may be defined as the characteristic surface configuration of an object, an outline or contour. It permits an object to be distinguished from its surroundings by its outline[12]. We have considered edge detection technique for extracting Shape representation of an image. we are using prewitt filter to determine the horizontal and vertical edges of the query image [1]. Boundary-based shape representation only uses the outer boundary of the shape. This is done by describing the considered region using its external characteristics i.e. the pixels along the object boundary. Changes or discontinuities in an image amplitude attribute such as luminance value are fundamentally important primitive characteristics of an image because they often provide an indication of the physical extent of objects within the image. Local discontinuities in image luminance from one level to another are called luminance edges [1].
(4.4)
(4.5)
This mask is used for Prewitt filter which we have used for calculating the boundary feature of the image.
40
We can calculate the horizontal and vertical edges separately.Then we can calculate the combined edges from these two edges. It is given as [1] Combined edges = Where, Icx=filtered result of image data with mask Px, Icy=filtered result of image data with mask Py.
(4.6)
Here the order used is 2.5 , as we go on increasing the order the number of edges become more as it will represent the small change in the pixel to pixel value. So the edges will be more clear but for the analysis of image retrieval that much accuracy is not necessary. So the order between 2 to 4 is sufficient.
41
Algorithm for calculating shape feature [1]:
Input the Image
Filter the image using Prewitt filter for obtaining vertical edges.
Use complement of Prewitt filter for obtaining horizontal edges.
Calculate the combined edges of query image
Calculate the standard deviation of combined edges of query image
Use this as standard shape feature
Figure 4.12: Flowchart for calculating shape feature
42
4.3.1 RESULTS (shape feature : edge detection)

Here is one of the results using co-occurrence matrix matching algorithm. QUERY IMAGE:
RETRIEVED IMAGES :
43
DISCUSSION:
We have implemented boundary scanning technique (both horizontal and vertical) for extracting shape features of an image. Boundary scanning technique yields intended results for images having objects with specific shape. Hence it overcomes the drawback of colour and texture based retrieval.
44
4.4 INTEGRATION OF ALL FEATURE VECTORS

Each image feature such as colour, texture and shape is suitable for retrieval for a particular class of images. Hence to implement efficient CBIR system for general purposes, we have gone for integration of all four feature vectors viz. colour, cooccurrence matrix, wavelet packet decomposition, edge detection. We have each particular feature vector of following length: Colour Feature Vector = 20; Co-occurrence matrix Feature Vector =25; Wavelet Packet Decomposition Feature Vector =12; Edge Detection Feature Vector =25. After INTEGRATION ,length of new vector will be:
45
Integrated Feature Vector = 82.
4.4.1 ASSIGNING DIFFERENT WEIGHT TO EACH INDIVIDUAL FEATURE. We have given different weight to each individual feature depending on users requirement. We multiply each individual feature vector by user specified weight. e.g. to give 35% weightage to colour feature we multiply it with 0.35 . In similar way, we give different weightage to remaining feature as per users requirement.
Figure 4.14: GUI designed for assigning weights
The importance of specific feature is increased by assigning more weight and similarly decreased by assigning lesser weight. If we assign 35% weight to colour feature, it means that colour feature is utilized upto 35% of its total strength. In this context if we assign 10% weight to co-occurrence matrix feature, then certainly colour feature gets more importance than co-occurrence matrix feature. Optimum weights are to be found empirically based on particular application.
4.4.2 G.U.I
The Graphical User Interface was constructed using MatLab GUIDE or Graphical User Interface Design Environment. Using the layout tools provided by GUIDE, we designed the following graphical user interface figure (gr_17_cbir.fig) for our CBIR application:
46
Figure 4.15: Developed GUI in MATLAB GUIDE
4.2.4.1 Query Image selection:

Query Image can be selected by clicking Select pushbutton. Any image from computer or external device connected to the computer can be selected as a query image.
47
Figure 4.16:Selection of query-image
4.2.4.2 Adjusting feature weights:

Feature weights can be adjusted with the help of slider movement.
Figure 4.17:Adjuting appropriate feature weights
4.2.4.3 Retrieval of images :

Images similar to the queryimage can be retrived byclicking Search pushbutton.Top six results are displayed.
48
Figure 4.18:Image retrieval by clicking on search pushbutton.
4.5
Relevance feedback:
Specification of the query is very critical in CBIR. In fact, if the sample image is located near to the boundary of the relevant cluster in the feature space, the first retrieval may contain few relevant images. In this case, the query refinement mechanism is useful to move the query towards the middle of the cluster of relevant images[19].We use the sets of relevant and non-relevant images ( calculate a new query and
; ) specified by the user to
by applying the Rocchios formula [19] (4.7)
Where Q is the previous query,
are the number of images in
;and
are the feature vectors associated to the relevant and irrelevant images, , , and parameters [with ] control the relative
respectively. In (4.7),
importance of the previous query, the relevant images and the nonrelevant images, respectively[19].
49
One of the results we have got is as follows
Figure 4.19:One of the results Relevance feedback can be given by ticking the checkbox named feedback. User is asked about the most similar image he /she thinks through command window when he/she click on Search pushbutton.User has to the rank of the most similar image.
Figure 4.20: Users feedback through command prompt
After query refinement we got following results.
50
Figure 4.21: Results after giving relevance feedback
4.6 DATABASE
The image database that we used in our project contains 350 JPG images. Few images are photographs of S.T.E.S. campus taken from digital camera. Few images are photographs of AutoExpo.in Delhi taken from digital camera. Remaining images have been randomly selected from the World Wide Web.
4.7 ACCOMPLISHMENTS Demo 1.
51
Figure 4.21: Demo of retrieval(1)
Demo 2.
Demo 3.
Here query image is an image drawn in Paint software
52
Figure 4.24: Demo of retrieval(3)-improved results by assigning diff.weights.
Demo 4.
Here query-image is not in the database & is drawn with the help of Paint software.
53
Figure 4.25: Demo of retrieval(4) Results are improved by giving feedback about 5th image as relevant & 4th as irrelevant.
Figure 4.26: Improved results( by relevance feedback )
CONCLUSIONS
Based on implementation of CBIR system on MATLAB we came to following conclusion:
54
A. COLOUR:
Colour is a low level (pixel level) feature. It gives intended results for images containing natural beauty and similar kind of images but it doesnt produce intended search results for images having specific shape or structure.
Hence for building effective as well as efficient CBIR system, one should think beyond colour based image retrieval.
B. TEXTURE: Co-occurrence matrix based image retrieval yields good results for images containing certain structure. Wavelet packet decomposition based image retrieval yields even more intended results for images having specific structure. This was the drawback of colour based image retrieval which is removed by wavelet decomposition.
C. SHAPE: Boundary scanning technique yields intended results for images having objects with specific shape. Hence it overcomes the drawback of colour and texture based retrieval.
D. Each image feature such as colour, texture and shape is suitable for retrieval for a particular class of images. Hence not any one of the feature can be used as standalone feature for retrieval of images even though it produce excellent results for a particular class of images.
55
E. For building effective as well as efficient CBIR system we have gone for integration of all these features like colour, texture and shape. F. Different weights are applied to each feature as per its importance depending on user. G. Optimum weights for respective features are decided empirically. We have found optimum weights as:Colour:35%; Co-occurrence matrix:10%; Wavelet Feature:20% and Shape:35% as far as general applications are considered .
H. Depending on the user and nature of database optimum values of weights for respective image features can be different e.g. if database contains images of historical importance having artistic drawings and constructions, then greater weight should be assigned to shape and texture rather than colour.
I. Query refinement by Rocchios formula helps in moving query towards the middle of the cluster of relevant images. Hence relevance feedback using query refinement technique improves results.
FUTURE ASPECT
CBIR is still a developing science. As image compression, digital image processing, and image feature extraction techniques become more developed, CBIR maintains a steady pace of development in the research field. Current techniques are based on low level features and there is a huge semantic gap existing. Much more research work is needed for coming out with a reliable and semantically competent system Furthermore, the development of powerful processing power and faster and cheaper memories contribute heavily to CBIR development. This development promises an immense range of future applications using CBIR.
56
REFERENCES
1. K. Jalaja, Chakravarthy Bhagvati, B. L. Deekshatulu, Arun K. Pujari Texture
Element Feature Characterizations for CBIR:-Geoscience and Remote Sensing Symposium,2005. IGARSS 05. Proceedings 2005 IEEE International Volume 2, Issue , 25-29 July 2005 . IEEE CNF
2. Rudinac.S.; Zajic.G.; Uscumlic.M.; Rudinac.M.; Reljin.B.:- Comparison of CBIR
Systems with Different Number of Feature Vector Components ;Semantic Media Adaptation and Personalization, Second International Workshop on 17-18 Dec. 2007. IEEE CNF 3. Barbeau Jerome, Vignes-Lebbe Regine, and Stamon Georges, A Signature based on Delaunay Graph and Co-occurrence Matrix, Laboratoire InformatiqueA et Systematique, Universiyt of Paris, Paris, France, July 2002.
4. K.P.Soman,K.I.Ramchandran ;Insight into Wavelets-From Theory to Practice ;
PHI publication,2003 edition.

5. Linda G. Shapiro, and George C. Stockman, Computer Vision, Prentice Hall,
2001.
6. R. Jain, R. Kasturi, and B. G. Schunck, Machine Vision, McGraw Hill
International Editions, 1995.

7. FOLDOC, Free On-Line Dictionary Of Computing, texture, May 1995, [Online
Document]
8. Texture, class notes for Computerized Image Analysis MN2, Centre for Image
Analysis, Uppsala, Sweden, Winter 2002

9. FOLDOC, Free On-Line Dictionary Of Computing, wavelet, May 1995, [Online
Document]
10. Lexico Publishing Group, LLC, shape, [Online Document] 11. Benjamin B. Kimia, Symmetry-Based Shape Representations, Laboratory for
Engineering Man/Machine Systems (LEMS), IBM, Watson Research Center, October 1999.
57
12. FOLDOC, Free On-Line Dictionary Of Computing, co-occurrence
matrix, May 1995, [Online Document] 13. Stephen J. Chapman; MATLAB Programming for Engineers: second edition, Bookman companion series. 14. HuiA Yu1, Mingjing Li2, Hong-Jiang Zhang2, Jufu Feng1;Color texture moments for content-based image retrieval* Center for Information Science, Peking University, Beijing 100871, China ,Microsoft Research Asia,49 Zhichun Road, Beijing 100080 , China
15. Gonzalez & Woods,Digital Image Processing using MATLAB Pearson
Education, Third Edition,2009 reprint.

16. Remco C. Veltkamp, Mirela Tanas ; Content-Based Image Retrieval Systems: A
Survey; Department of Computing Science, Utrecht University 17. John Eakins,Margaret Graham; Content Based Image Retrieval; University of Northumbria at Newcastle ;Report: 39-JISC Technology Applications Program 18. Shengjiu Wang, A Robust CBIR Approach Using Local Color Histograms, Department of Computer Science, University of Alberta, Edmonton, Alberta, Canada, Tech. Rep. TR 01-13, October 2001. 19. Anelia Grigorova, Francesco G. B. De Natale, Charlie Dagli, and Thomas S. Huang; Content Based Image Retrieval by Feature Adaptation and Relevance Feedback. IEEE Transactions on multimedia, vol. 9, no. 6, October 2007
PUBLICATIONS:
T.D.Joshi,V.K.Kolhe,A.B.Kore;Implementation of common CBIR techniques, A National conference on pervasive computing, 9-10 April 2010,Dept. of Electronics & Telecommunication, Sinhgad college of Engineering, Pune41,Maharahtra,India.
58

Project Report New 03

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Project Report New 03

Transféré par

Droits d'auteur :

Formats disponibles

1

1.1.1 Applications of CBIR Examples of CBIR applications are [17]:

Crime prevention: Automatic face recognition systems, used by police forces.

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

1.2 PROBLEM MOTIVATION

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

CHAPTER 2 LITERATURE SURVEY

Figure 2.1(a): Sample Image (Source: Internet)

Figure 2.1(b): H,S,V components of sample image

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

(b) Bricks Figure 2.2: Examples of Textures. (Source: Internet)

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

C is called a co-occurrence matrix defined by P.

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Figure 2.3: Image example (8 grey-level image)

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

entry in the matrices, represents the probability of

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Mathematically, we denote a wavelet as [4]

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Figure 2.5 : Haar Wavelet Example [4]

Haar wavelet scaling function is defined as:

0 <= t <= <=t<=1 otherwise

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Figure 2.6 : Daubechies Wavelet Example (Source: Internet)

Daubechies wavelet scaling function is defined as:

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Figure 2.7: Boundary-based & Region-based Shape representations(Source: Internet)

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

CHAPTER 3 CBIR METHODOLOGY

Feature Vectors Similarity Comparison

Visual Content Description

Feature Database Indexing & Retrieval

Figure 3.1: CBIR Methodology

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

IMPLEMENTATION & RESULTS

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

CHAPTER 4 IMPLEMENTATION & RESULTS 4.1 COLOUR

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Compute the histogram for each plane

Show retrieved images

Figure 4.1 : Flow of implemented image retrieval system

4.1.1 RESULTS (colour)

Figure 4.2(a):A Query Image(1) (Source: Internet)

Figure 4.2(b): Retrieved Images for query image(1)

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Figure 6.2(b): Retrieved Images for query image(1)

Figure 4.3(a):A Query Image(2) (Source: Internet)

Figure 4.3(b): Retrieved Images for query image(2)

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Co-occurrence Matrix Wavelet Packet Decomposition

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

Table 4.1: Offset values for corresponding angles

Determine standard deviation for each column of co-occ matrix

Compare standard deviation vectors

Show retrieved images