2010 Second International Conferences on Advances in Multimedia

Comparison of Two Image Segmentation Algorithms

Alina Doringa, Gabriel Mihai
University of Craiova, Craiova Bd. Decebal 107, Romania alinadoringa@hotmail.com mihaicristiangabriel@hotmail.com Abstract Image segmentation has been, and still is, a relevant research area in computer vision and hundreds of segmentation algorithms have been proposed in the last 30 years. An image retrieval system has now an array of available algorithm choices; however, few objective numerical evaluations of these segmentation algorithms exist. Typically, the effectiveness of a new algorithm is demonstrated only by the presentation of a few segmented images and an expert decides the effectiveness of a technique based on application requirements. This paper presents a comparison between two image segmentation algorithms: the color set back-projection algorithm that can be found in many related studies and an original segmentation method using a hexagonal structure defined on the set of image pixels. For this purpose error measures are used. The error measures quantify the consistency between these two segmentations algorithms.
Keywords-component: Image segmentation; Segmentation evaluation; Segmentation comparison, Error measures.

Liana Stanescu, Dumitru Dan Burdescu

University of Craiova, Craiova Bd. Decebal 107, Romania {stanescu_liana, burdescu_dumitru}@software.ucv.ro

I. INTRODUCTION Image segmentation algorithms have matured to the point that they provide segmentations which agree to a large extent with human intuition. These segmentations play a large role in object recognition. It is clear that segmentation can be used to help cue and refine various recognition algorithms. A potential problem for a measure of consistency between segmentations is that there is no unique segmentation of an image. For example, two people may segment an image differently because either they perceive the scene differently, or they segment at different granularities. If two different segmentations arise from different perceptual organizations of the scene, then it is fair to declare the segmentations inconsistent [3]. This paper presents an objective and quantitative study of two segmentation algorithms that will be described further. The color set back-projection is a well known algorithm that can be found in many related studies. This method was implemented and tested on a wide variety of images and have achieved good results in automated detection of color regions from an image. The second
978-0-7695-4068-9/10 $26.00 2010 IEEE DOI 10.1109/MMEDIA.2010.25 185

method chosen is our original algorithm and the novelty of this algorithm consists of: the hexagonal structure used in the unified framework for image segmentation, the using of maximum spanning trees for determining the set of nodes representing the connected components, and the efficient method for extracting the boundary of visual objects from connected components representing salient objects of an image. For each of these algorithms, three characteristics are examined [4]: 1. Correctness: the ability to produce segmentations which agree with human intuition. That is, neither segmentations which correctly identify structures in the image at neither too fine nor too coarse a level of detail. 2. Stability with respect to parameter choice: the ability to produce segmentations of consistent correctness for a range of parameter choices. 3. Stability with respect to image choice: the ability to produce segmentations of consistent correctness using the same parameter choice on a wide range of different images. The comparison of these two image segmentation algorithms is based on two metrics (GCE, LCE) defined in [3] which can be used to evaluate the consistency of a pair of segmentations. These measures allows a principled comparison between segmentation results on different images, with differing numbers of regions, and generated by different algorithms with different parameters. The segmentation results from these two algorithms are compared to the manual segmentations of the same image. The paper is organized as follows: in Section 2 related work is briefly presented, Section 3 describes the color set back-projection algorithm; Section 4 presents the method for construction of hexagonal structure on image pixels image and the new algorithm for segmentation of images represented in a HSV color space; Section 5 describes the two error metrics; Section 6 presents the experimental results; Section 7 concludes the paper. II. RELATED WORK There is a large literature on segmentation and clustering, dating back over 30 years, with applications in many areas other than computer vision.

In this section it is presented some of the related work that is most relevant to the approach of the paper. Most graph-based segmentation methods attempt to search a certain structures in the associated edge weighted graph constructed on the image pixels, such as minimum spanning tree [6] [7], or minimum cut [8] [9]. The major concept used in graph-based clustering algorithms is the concept of homogeneity of regions. For color segmentation algorithms the homogeneity of regions is color-based, and thus the edge weights are based on color distance. Early graph-based methods use fixed thresholds and local measures in finding segmentation. To overcome the problem of fixed threshold, in [10] is determined the normalized weight of an edge by using the smallest weight incident on the vertices touching that edge. Other methods presented in [6] [7] use an adaptive criterion that depends on local properties rather than global ones. The methods based on minimum cuts in a graph are designed to minimize the similarity between pixels that are being split [8], [9]. An alternative to the graph cut approach is to look for cycles in a graph embedded in the image plane [11]. Most graph-based segmentation approaches use color and texture models. However these homogeneity criteria have some drawbacks for object extraction. In [12] a source of additional information denoted by the term of syntactic features is presented. This information represents geometric properties of regions and their spatial configurations. Examples of such features include homogeneity, compactness, regularity, inclusion or symmetry. In this paper, two graph-based segmentation methods, the color set back-projection method and our original segmentation method using a hexagonal structure defined on the set of image pixels, are compared.
III.

THE COLOR SET BACK-PROJECTION ALGORITHM

The color set back-projection algorithm proposed in [5] is the technique implemented for the automated extraction of regions and representation of their color content. The extraction system for color regions has four steps: 1. the image transformation, quantization and filtering (the transformation from the RGB color space to HSV color space and the quantization of the HSV color space at 166 colors) 2. back-projection of binary color sets 3. the labeling of regions 4. the extraction of the region features The algorithm follows the reduction of insignificant color information and makes evident the significant color regions, followed by the generation, in automatic way, of the regions of a single color, of the two colors, of three colors. To conclude with the second step of the color set back-projection algorithm, is the following [5]: 1. Detection of single color regions 1.1. Having the image histogram, H[m], all the values m'=m for which H[m]>=p0 are detected.

1.2. For each m' the color set c having the property c[k]=1 for k=m and c[k]=0 in other cases is found. On the image R[m,n] the back-projection algorithm for each color set c is applied and the color regions are found. For each region n the local histogram Ln[m] is stored. 1.3. The residue histogram Hr[m]=H[m]-nLn[m] is computed. 2. Detection of two colors regions 2.1. The values l'=l and m'=m, l m, H[l]>=p0, H[m]>=p0 and Hr[l]>=p1, Hr[m]>=p1 are found. 2.2. For each set l', m' the color set c having the property c[k]=1 for k=l' or k=m' and c[k]=0 in other cases is found. On the image R[m,n] the back-projection algorithm for each set c is applied and the color regions are found. For each region the local histogram Ln[m] is recorded. 2.3. The residue histogram Hr[m]=H[m]-nLn[m] is updated. 3. Detection of the three colors regions, etc. For each detected regions the color set that generated it, the area and the localization are stored. All the information is necessary further on for the content-based region query with absolute or relative localization. The region localization is given by the minimal bounding rectangle. The region area is represented by the number of color pixels, and can be smaller than the minimum bounding rectangle. In the implementation of the color set back-projection algorithm that can be found it in [1] [13], the image can be read in a .bmp format. Each pixel from the initial image is transformed in HSV format and quantized. At the end of this processing both the global histogram of the image, and the color set are available. On the matrix that memorizes only the quantized colors from 0 to 165 is applied a 5x5 median filter, which has the role of eliminating the isolated points. Having the HSV quantized matrix it is possible to begin the process of regions extraction presented above. In the implementation it may be observed that this process is in fact a depth first traversal, described in pseudo-code in the following way: procedure FindRegions (Image I, colorset C) InitStack(S) Visited = for *each node P in the I do if *color of P is in C then PUSH(P) Visited Visited {P} while not Empty(S) do CrtPoint <- POP() Visited Visited {CrtPoint} For *each unvisited neighbor S of CrtPoint do if *color of S is in C then Visited Visited {S} PUSH(S)

186

//end if //end for //end while *Output detected region //end if //end for The total running time of a call of the procedure FindRegions (Image I, colorset C) is O (m2*n2), where m is the width and n is the height of the image [1][13].
IV. IMAGE SEGMENTATION USING A HEXAGONAL STRUCTURE
DEFINED ON THE SET OF PIXELS

Figure1. Hexagonal structure constructed on the image pixels

The technique is based on a new and original utilization of pixels from the image that are integrated into a network type graph [2]. A hexagonal network structure on the image pixels, as presented in Figure 1, is used, in order to minimize the running time. Each point which forms an image on the screen must be properly addressed in order to be indexed. The disposition of the points on the plane, can take different choices. Considering technical implementation, these points must be placed as regularly as possible on the plane and they must be disposed so that the coverage of the plane is as efficient as possible. There exist only three possible regular tessellation schemes to tile a plane without overlapping among the samples and gaps between them, namely the tessellation with hexagons, with squares, and with regular triangles. Any other types of spatial tessellation will result in either unequal distance between neighboring pixels, or introduce gaps or overlaps among samples. The vertices of the hexagonal network cover half of pixels from the image. The hexagonal structure represents a grid-graph, for each hexagon h in this structure there exist 6hexagons that are neighbors in a 6-connected sense and the determination of indices for 6-hexagons neighbors having as input the index of current hexagon is very simple. The main advantage when using hexagons instead of pixels as elementary piece of information is the reduction of the time complexity of the algorithms. The list of hexagons is stored such as a vector of numbers [1..N], where N, the number of hexagons, is determined based on the formula [2]:
N = height 1 width width mod 4 width width mod 4 4 * + 2 4 4

The pixels of image are split into two sets, the set of pixels which represent the vertices of hexagons and the set of complementary pixels; the two lists will be used as inputs for the segmentation algorithm. The mapping of pixels network on the hexagons network is not time consuming. The proposed segmentation algorithms will produce a proper segmentation of pixels which are mapped on the hexagon network according to the definition of distance in HSV color space and the syntactic features for each region. Algorithms for regions list determination These algorithms proposed in [2] return a composed list corresponding to the salient colors from the input image, which is computed based on the hexagonal network and the distance between two colors in HSV space color. The procedure sameVertexColour returns the color of a hexagon. This procedure has a constant execution time because all calls are constant in time processing. A. Determination of the hexagon color Input: The current hexagon hi . The colors list of pixels corresponding to the hexagonal network and colors list for complementary pixels of c c L1 = { p1 ,........... p6 n } , L2 = { p1 ,.........., p6 n } Output: The object crtColorHexagon Procedure sameVertexColour ( hi , L1 , L2 ; crtColorHexagon ); initialize crtColorHexagon; determine the colors for the six vertices of hexagon h i ; determine the colors for the two vertices from interior of hexagon hi ; calculate the mean color value meanColor for the eight colors of vertices; crtColorHexagon.colorHexagon meanColor; crtColorHexagon:sameColor true; for k to 6 do 1 if colorDistance(meanColor, colorVertex[k]) > threshold then crtColorHexagon:sameColor false; break; end end return crtColorHexagon;

with height and width representing the height and the width of the image. Each hexagon has associated two important attributes representing its dominant color and its gravity center. For determining these attributes there are used eight pixels contained in a hexagon: the six pixels of the frontier, and the two interior pixels. The dominant color of a hexagon is the main vector color of all the eight colors of its associated pixels.

187

Expand the current region Input: The current hexagon h i The colors list of pixels corresponding to the hexagonal network and colors list for complementary pixels of L1 = { p1 ,........... p6 n } , L2 = { p1c ,.........., p6cn } The current region index indexCrtRegion and the current color index indexCrtColor Output: The list of hexagons with the same color crtRegionItem Procedure expandColourArea ( hi , V , L1 , L2 , indexCrtRegion, indexCrtColor; crtRegionItem); push( hi ); while not(empty(stack)) do h pop(); determine the six hexagons neighbors nHexagons for hexagon h; for k 1 to 6 do if not(visit(nHexagons[k])) then crtColorHexagon sameVertexColour( hi , L1 , L2 ); if crtColorHexagon.sameColor and colorDistance(crtColorHexagon.color;colors[index CrtColor]) < threshold) then add nHexagons[k] to crtRegionItem; mark visit(nHexagons[k]); push (nHexagons[k]); end end end end B. Determination of the regions list Input: The vector corresponding to the hexagonal network V = {h1 ,......hn }.The colors list of pixels corresponding to the hexagonal network and colors list for complementary pixels of L1 ; L1 = { p1 ,........... p6 n } , L2 = { p1c ,.........., p6cn } Output: A list of colors pixels, for each color we determine a list of regions C = {{c1},........, {ck }},

indexCrtRegion 0; else indexCrtColor k; indexCrtRegion findLastIndexRegion (indexCrtColor); indexCrtRegion++; end hi .indexRegion indexCrtRegion;

hi .indexColor
add

k;

hi to crtRegionItem; expandColourArea( hi , L , L2 ;V,indexCrtRegion,

1

indexCrtColor; crtRegionItem); add new region crtRegionItem to list of element k from C end end end The running time of the procedure listRegions is O( n ), where n is the number of the hexagons network [2].
2

V. SEGMENTATION ERROR MEASURES

There is usually more than one way to measure accuracy, precision and performance for a segmentation algorithm. A potential consumer of an algorithms output needs to know what types of incorrect/invalid results to expect, as some types of results might be acceptable while others are not. Thus multiple metrics are necessary for potential consumers to make intelligent decisions. In [3] two metrics that can be used to evaluate the consistency of a pair of segmentations are proposed. Segmentation is simply a division of the pixels of an image into sets. A segmentation error measure takes two segmentations S1 and S 2 as input, and produces a real valued output in the range [0..1] where zero signifies no error. A measure of error at each pixel it is defined. This measure is tolerant to refinement as the basis of both measures. For a given pixel pi consider the segments in

S1 and S 2 that contain that pixel. The segments are sets of

pixels. If one segment is a proper subset of the other, then the pixel lies in an area of refinement, and the local error should be zero. If there is no subset relationship, then the two regions overlap in an inconsistent manner. In this case, the local error should be non-zero. Let \ denote set difference, and |x| the cardinality of set x. If R(S; p i ) is the set of pixels corresponding to the region in segmentation S that contains pixel p i , the local refinement error is defined in [3] as:
E (S 1, S
2

ci = {color , {r1 ,....., rp }}

Procedure listRegions (V , L1 , L2 C); colourNb 0; for i 1 to n do initialize crtRegionItem; if not(visit(hi)) then crtColorHexagon sameVertexColour ( L1 , L2 ; C ); if crtColorHexagon.sameColor then k findColor(crtColorHexagon.color); if k < 0 then add new color k

, p i) =

| R (S 1, p i) \ R (S 2 , p i) | | R (S 1, p i) |

colourNb++;

ccolourNb

to list C;

Note that this local error measure is not symmetric. It encodes a measure of refinement in one direction only. E( S1 ; S 2 ; pi ) is zero precisely when S1 is a refinement of

188

S 2 at pixel pi , but not vice versa. Given this local

refinement error in each direction at each pixel, there are two natural ways to combine the values into an error measure for the entire image. Global Consistency Error (GCE) forces all local refinements to be in the same direction. Local Consistency Error (LCE) allows refinement in different directions in different parts of the image. Let n be the number of pixels [3]: 1 GCE ( S 1 , S 2 ) = min E ( S 1 , S 2 , p i ), E ( S 2 , S 1 , p i ), i i n 1 LCE ( S 1 , S 2 ) = min {E ( S 1 , S 2 , p i ), E ( S 2 , S 1 , p i ) } n i As LCE GCE for any two segmentations, it is clear that GCE is a tougher measure than LCE. In [3] it is shown that, as expected, when pairs of human segmentations of the same image are compared, both the GCE and the LCE are low; conversely, when random pairs of human segmentations are compared, the resulting GCE and LCE are high.

Table II shows the number of regions resulted from the application of segmentation using a hexagonal structure and also the values of error measures (GCE and LCE):
TABLE II. EXPERIMENTAL RESULT FOR THE SEGMENTATION USING A HEXAGONAL STRUCTURE

Image No. 1

Image

Regions No 3

GCE

LCE

0.09

0.07

0.10

0.12

0.09

0.07

VI. EXPERIMENTAL RESULTS In this section the regions obtained by applying both segmentation algorithms: the color set back-projection segmentation and the segmentation using a hexagonal structure defined on the set of pixels and error measures values are presented. Because human segmentation it is considered truth segmentations, the error measures are calculated in relation with manual segmentation. The experiments were made on a set of 200 images. It should be noted that both algorithms use a fixed threshold value of 512. This means that only the regions that have a number of pixels greater than this value will be kept. Table I shows the number of regions resulted from the application of the color set back-projection algorithm and the values of error measures (GCE and LCE).
TABLE I.
EXPERIMENTAL RESULTS FOR THE COLOR SET BACK-PROJECTION ALGORITHM

The following are the regions resulted from manual segmentation and from the application of the two methods presented above. Image No.1:

Fig.2 Regions from manual segmentation

Fig.3 Regions from the color set back-projection algorithm

Image No. 1

Image

Regions No. 4

GCE

LCE
Fig.4 Regions from hexagonal structure segmentation

0.18

0.11

Image No.2:

0.36

0.18
Fig.5 Regions from manual segmentation

0.18

0.17
Fig.6 Regions from the color set back-projection algorithm

189

Fig.7 Regions from hexagonal structure segmentation

Image No.3:

Fig.8 Regions from manual segmentation

Fig.9 Regions from the color set back-projection algorithm

algorithm produces less consistent regions than the second algorithm. Then for evaluate the accuracy of the segmentation are used error measures. The proposed error measures quantify the consistency between segmentations of differing granularities. Because human segmentation is considered truth segmentations the error measures are calculated in relation with manual segmentation. For the two errors measured, it was demonstrated that both algorithms can create realistic segmentations with a wide variety of parameters. The GCE and LCE demonstrate that the image segmentation based on a hexagonal structure produces a better segmentation than the back-projection method. In the future work the comparative study will be effectuated on medical images. The comparison will be extended also to a wide range of graph-based segmentation methods. VIII. REFERENCES
[1] D. D. Burdescu, L. Stanescu, A new algorithm for content-based region query in multimedia databases Congres DEXA 2005, Database and expert systems applications Copenhagen, 22-26 August 2005 , vol. 3588, pp. 124-133. D. D. Burdescu, M. Brezovan, E. Ganea, and L. Stanescu, A new method for segmentation of images represented in a HSV color space ACIVS 2009, Bordeaux, France, pp. 606-617 . D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics in IEEE (ed.), Proceedings of the Eighth International Conference On Computer Vision (ICCV-01), July 7-14, 2001, Vancouver, British Columbia, Canada, vol. 2, pp. 416425. R. Unnikrishnan, C. Pantofaru, and M. Hebert, Toward objective evaluation of image segmentation algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 6, June, 2007, pp. 929-944. J. R. Smith, S. F. Chang, Tools and techniques for color image retrieval in Science and Technology - Storage & Retrieval for Image and Video Databases IV, volume 2670, San Jose, CA, February 1996. IS&T/SPIE. (1996) P.F. Felzenszwalb, W.D. Huttenlocher, Efficient graph-based image segmentation in Journal of Computer Vision, 59(2), pp. 167181 (2004) L. Guigues, , L.M Herve, L.-P. Cocquerez, The hierarchy of the cocoons of a graph and its application to image segmentation. Pattern Recognition Letters, 24(8), pp. 10591066 (2003) J. Shi, J. Malik, Normalized cuts and image segmentation, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 731737 (1997) Z. Wu ,R. Leahy, An optimal graph theoretic approach to data clustering: theory and its application to image segmentation, IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(11),pp. 11011113 (1993) R. Urquhar, Graph theoretical clustering based on limited neighborhood sets. Pattern Recognition, 15(3), pp 173187 (1982) I. Jermyn, H. Ishikawa, Globally optimal regions and boundaries as minimum ratio weight cycles. IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(8), pp. 10751088 (2001) C.F. Bennstrom, J.R. Casas, Binary-partition-tree creation using a quasi-inclusion criterion. Proc. of the Eighth International Conference on Information Visualization, London, UK, pp. 259294, (2004) L. Stanescu Visual information. Procecessing, Retrieval and Applications Sitech Craiova 2008

Fig.10 Regions from hexagonal structure segmentation

[2]

If two different segmentations arise from different perceptual organizations of the scene, then it is fair to declare the segmentations inconsistent. If, however, segmentation is simply a refinement of the other, then the error should be small, or even zero. The error measures presented in the above tables are calculated in relation with manual segmentation which is considered truth segmentations. From the two tables it can be observed that the values for GCE and LCE are lower in case of hexagonal segmentation. For example for Image No.1 GCE = 0.09 and LCE =0.07 in the case of hexagonal segmentation and GCE=0.18 and LCE=0.11 for the color set back-projection algorithm. The error measures, for almost all tested images, have smaller values in case of the original segmentation method which use a hexagonal structure defined on the set of pixels. Because the error measures for segmentation using a hexagonal structure defined on the set of pixels are lower than color set back-projection segmentation is clearly inferred that the segmentation method based on hexagonal structure is more efficient than the other. Experimental results show that the original segmentation method based on a hexagonal structure is a good refinement of the manual segmentation. VII. CONCLUSION This paper presented a comparison of two image segmentation algorithms: the color set back-projection algorithm and the image segmentation using a hexagonal structure defined on the set of image pixels. It is compared first the correctness of segments resulted applying the two algorithms described above. Both algorithms have the ability to produce segmentations that agree with human intuition. It is evident that the color set back-projection

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10] [11]

[12]

[13]

190