Académique Documents
Professionnel Documents
Culture Documents
-------------------------------------------------------- ABSTRACT-----------------------------------------------------------------
Multimedia database consist of media data types such as text, images, sound, and video, which are retrieved by
image descriptors such as texture, shape, and color as their primary key. New technologies have numerous
challenges, and multimedia database have its share of challenges. Some of these challenges are in Content-based
Image Retrieval (CBIR) techniques, such as irregular performance measurements in motion, location and sketches,
hence creating gap in multimedia database evaluation techniques. As a result of which, this paper was motivated to
close this gap of lack of an effective and precise performance evaluation benchmarking measure. To address the
irregular performance measure, the paper developed a performance measure benchmarking model (PMBM)
framework using image descriptors in query by image content (QBIC). Moreover, the paper adopted the de Groots
empirical research cycle methodology by implementing the five stages methodology of: image preparation, query
definition, data collection, data evaluation and framework building for the PMBM framework development.
Furthermore, the paper conducted several experiments on texture, color and shape image datasets with respect to
performance measures of accuracy, recall, precision and F-Measures on CBIR DB2 database. The results of these
experiments were used to develop the PMBM framework and were analyzed by a statistical software SPSS version
20. Even more importantly, the paper developed a software evaluation tool in JAVA programming language from
the PMBM framework to qualify and quantify its effectiveness to measure performance of the existing CBIR
database systems. The results of this evaluation showed that the open source CBIR database FIRE was ranked as
High with baseline score value (BSV) of 0.92 (92%) and IMG(Anaktisi) was ranked as Low, with BSV of 0.87
(87%) from BSV of 0.90 (90%) of IBM DB (benchmark system).
Keywords: Content Based Image Retrieval (CBIR), Query by Image Content (QBIC), Image Descriptors,
Multimedia Databases and Performance Measure Benchmarking Modal (PMBM) Framework
-----------------------------------------------------------------------------------------------------------------------------
Date of Submission: 14 May 2017 Date of Accepted: 09 June 2017
------------------------------------------------------------------------------------------------------------------------------------------
I. INTRODUCTION
1.1 Background Information
Multimedia databases store, manage and retrieve media data type such as images, sounds and videos, while
traditional database store alphanumeric data types. The traditional database retrieve data through alphanumeric
unique keys known as primary keys, while multimedia databases use content based image retrieval (CBIR) such as
texture, color and shape as unique keys for retrieving data. CBIR is also known as query by image content (QBIC)
and content-based visual information retrieval (CBVIR), is an application of computer vision techniques for
searching and retrieving digital images in large databases [4]. In recent times, extensive research works have been
done on CBIR systems, where the emergence of the first commercial CBIR system an International Business
Machine (IBM) QBIC system was developed as the benchmarking system. However, it is observed that the many
systems use different standards of retrieval techniques such as motions, contours and sketches of various objects to
retrieve data [7]. This is evident by the proposed evaluation frameworks by [1] and [4], where their shortcomings to
address irregular performance measurement were also elusive. The above discussion infers biasness in multimedia
database system evaluation, and hence creating a gap, which is the motivation of this paper to close this gap by
developing a performance measure benchmarking model framework (PMBM) using image descriptors in QBIC.
The following equations were used to calculate the four performance metrics of accuracy, precision, recall,
and F-Measure during the experiments [10].
These performance measurements metrics were used for evaluation and validation of the proposed PMBM
Framework development.
ii. Established ground-truth human judges by randomly selecting 15 experts and novices from various university
departments who have no prior knowledge of CBIR technique to perform visual retrieval of image performance
measure
iii. Established set of 15 queries to be used by the 5 experts human judges during validation
iv. Computed and analyzed the results to build the PMBM framework.
3.1 Introduction
The paper conducted three experiments using three datasets of: Color Image Dataset, Texture Image Dataset and
Shape Image Dataset and were evaluated against four performance metrics of: Accuracy, Precision, Recall and
F-Measure in order to develop the PMBM Framework
These results inferred that the DB2 CBIR database using color descriptors correctly retrieved about three quarters of
the query and only one quarter was incorrectly retrieved as shown in Figure 2. However, these results are not
conclusive since there are other factors contributing to the performance of the retrieval process. Therefore, the paper
computed four performance metrics of: Accuracy, Precision, Recall and F-Measure to justify these results. The
results showed that; the accuracy = 83%, precision = 88%, recall = 91% and F-Measure = 89%. Finally, the
computed mean score for color descriptor was 87.75%.
These results inferred that the DB2 CBIR database using texture descriptors correctly retrieved more than three
quarters of the query and only less than a quarter was incorrectly retrieved as shown in Figure 3. However, these
results are not conclusive since there are other factors contributing to the performance of the retrieval process.
Therefore, the paper computed four performance metrics of: Accuracy, Precision, Recall and F-Measure to justify
these results. The results showed that; the accuracy = 87%, precision = 90%, recall = 94% and F-Measure = 92%.
Finally, the computed mean score for texture descriptor was 90.75%.
These results inferred that the DB2 CBIR database using shape descriptors correctly retrieved more than three
quarters of the query and only less than a quarter was incorrectly retrieved as shown in Figure 4. However, these
results are not conclusive since there are other factors contributing to the performance of the retrieval process.
Therefore, the paper computed four performance metrics of: Accuracy, Precision, Recall and F-Measure to justify
these results. The results showed that; the accuracy = 87%, precision = 90%, recall = 95% and F-Measure = 93%.
Finally, the computed mean score for shape descriptor was 91.25%.
The mean-scores for the three CBIR were computed and results showed that; 0.90 (90.0%) mean-score for IBM
DB2 ranked as Baseline, 0.92 (92.0%) for FIRE ranked as High from the Baseline and 0.87 (87.0%) for
IMG(Anaktisi) ranked as Low from the Baseline. These results are shown in Figure 8: CBIR Systems Performance
Evaluation. These results inferred that the PMBM Framework Software Tool evaluated the performance of FIRE as
92.0% and IMG(Anaktisi) as 87.% and ranked them as High and Low respectively from the Baseline value.
VI. RECOMMENDATION
Even though the findings of the paper showed that, the PMBM framework implementation was effective to
measure the performance of existing CBIR systems using descriptors of: texture, color and shape, an enhanced
version is recommended whereby more dynamic descriptors to be incorporated to increase effectiveness
of the framework. Finally, the paper also recommended the extension-ability of the PMBM framework using other
descriptors classification such as distributed based descriptors, differential descriptors and visual descriptors to give
it a wider scope.
REFERENCE
[1]. Christophe, J. (2012). Next Generation Search Engine: Advanced Models for Information Retrieval. Hershey, PA:
IGI Global. Retrieved from http://www.igi- global.com/book/next-generation-search-engines/59723
[2]. Cong, Y., Oliver, T., & Hassan, K. (2016). Stripes-based Object Matching In Studies in Computational Intelligence.
Springer. Retrieved from http://congyang.de/downloads.html
[3]. Kohavi, R., & Provost, F. (1998). Applied Research in Machine Learning. In Application of Machine Learning and
the Knowledge Discovery Process (Vol. 30). New York USA: Columbia University.
[4]. Lewandowski, D. (2012). A Framework for Evaluating the Retriev al Effectivenes s of Search Eng ines. IGI
Global.
[5]. Powers, D. (2011). Powers, David M W (2011). Evaluation: From Precision, Recall and F-Measure to ROC,
Informedness, Markedness & Correlation (PDF). Journal of Machine Learning Technologies. 2 (1): 3763. Journal
of Machine Learning Technologies, 2(1), 37 63.
[6]. Schaefer, G., & Stich, M. (2013). UCID - Uncompressed Colour Image Database. Retrieved from
http://homepages.lboro.ac.uk/~cogs/datasets/ucid/ucid.html
[7]. Subitha, S., & Suhatha, S. (2013). Survey paper on various methods in content based inforfmation retrieval.
IMPACT: International Journal of Research in Engineering and Technology, 1(3), 109120.
[8]. Weber, A. (2016). SIPI Image Database. Retrieved October 16, 2016, from http://sipi.usc.edu/database/
[9]. Wu, J., & M, J. (2010). CENTRIST: A Visual Descriptor for Scene Categorization. IEEE Transactions on Pattern
Analysis and Ma, 33(8), 1489 1501.
[10]. Zhu, M. (2004). Recall, Precision and Average Precision. . IMPACT: International Journal of Research in
Engineering and Technology