1 vues

Titre original : 00192473

Transféré par Kranthirekha Chennaboina

- A Survey of Methodaology of Fraud Detection Using Data Mining
- Clustering Presentation
- Cluster Analysis 1
- 541-693-1-PB
- Applying DM in ML
- DIP Final Project Report
- Enrollments Forecasting Based On Aggregated K-Means Clustering and Fuzzy Time Series
- Advance Quantitiative Thecnique
- IJETTCS-2013-10-25-084
- Report.docx
- 183800
- M.Tech
- Paper 28 - Clustering as a Data Mining Technique in Health Hazards of High Levels of Fluoride in Potable Water
- Fuzzy clustering Approach in segmentation of T1-T2 brain MRI
- Data Mining Practice Final Sol
- 10.1137@1.9780898718348.ch10
- Feature Subset Selection Techniques - Swift Clustering and Principle Component Analysis
- 2016 IEEE ns2 Projects| Dynamic Cluster Head Selection Method for Wireless Sensor Network
- adhoc
- daweqw

Vous êtes sur la page 1sur 9

T h e complete form of the transformation equations which map 112) M . Benard. "Extracting 3-D coordinates of an object from a digital

the scene coordinates onto the image coordinates is therefore stereopair: An automated method," in Proc. Signal Processing / I :

Throrir.c ~ j i dApplicurion.~,EUSIPCO. Erlangen, Germany. 1983. pp.

.Y' = (x + A.r) + b 227-230.

1131 H. H. Baker and T. 0. Binford, "Depth from edge and intensity based

and

stereo." in Proc. 7th Irzr. Joint Conf. Arrijcial Inrelligencr, vol. 2 ,

).' = (. . (.r + A.r) + d . (y + Ay) + e. 1981, pp. 631-636.

[ 141 B. Julesz. Foundurion.~ofCyclopean Perception. Chicago. IL: Uni-

In order to solve for the unknown parameters. we need to con- versity of Chicago Press, 1971.

sider the greatest source of random error. Due to the process of [IS] D. Marr and T. Poggio. "Cooperative computation of stereo dispar-

quantization involved in digitizing a video image, the measured ity," Science, v o l . 194, pp. 283-287, 1976.

image coordinates are going to b e in error. Thcrefore the unknown [ 161-. "A theory of human stereo vision," MlT Artificial Intell. Lab..

parameters can be computed by using a least squares techniques AI Memo45l. 1977.

which aims to minimize the difference between the measured image 1171 J . E. W . Mahew and J. P. Frisby. "The computation of binocular

edges," Perc.c~ptiotr,vol. 9, pp, 69-86, 1980.

coordinates and the computed image coordinates. A suitable tech-

I181 S . A. Lloyd. "A dynamic programming algorithm for binocular ste-

nique would be the generalized form of the Newton-Raphson reo vision," G E C J . R.. vol. 3, no. I , pp. 18-24, 1985.

method. The equations to solve are 1191 L. H. Quam, "Hierarchical warp stereo," in Proc. Darpa Image Un-

.Y;,) - (X + AX) - h = 0 dersranding Workshop, New Orleans. LA, 1984, pp. 21 1-215.

[20] D. C. Brown, "Calibration of close range cameras," in Proc. 121h

and Congress I n / . Soc. Phorogrammetry, Ottawa, Canada. 1972.

1211 -, "Analytical calibration of close range cameras,'' in Proc. SJtnp.

J,:, - C . (X + A.Y) - d . (J + AJ) - e =0 Close-Range Phorogrumtnetn, Melbourne, FL, 197 1 .

1221 K. W .Wong, Geomerric Culibrution o j TeletYsion Sysrems f o r Pho-

where (U,:,.y,:,) are the measured image coordinates. rogrummerric Applicuriorzs (Civil Eng. Studies, Photogrammetry Se-

T o obtain the reverse transform from the image coordinates ( x ' , ries no. 16). Urbana, IL: University of Illinois Press, 1968.

J' ) to the corrected image coordinates ( x , y ) . the image-to-image

mapping equations are reversed to provide the following transfor-

mation equations

.I- = (x' + h ' ) + Ax'

and

y = (c' . x' + d' . y ' + e ' ) + Ay' Unsupervised Optimal Fuzzy Clustering

where b ' . c'. d ' . and e' are constants and A x ' and A V ' arc system- I. GATH A ~ A.

D B. GEVA

distortion corrections.

Y . Shah. R. Chapman. R. Babaei-Mahani, and N . E. Lindsey. "Ex- guesses of cluster prototypes, and on assumptions made as to the num-

traction of range information from stereo images." Opt. LoscJrs E n g . . ber of subgroups present in the data. This study reports on a method

vol. 6. no. 2. pp. 125-127. 1985. for carrying out fuzzy classification without apriori assumptions on the

S . Nagata. "New versatile stereo (NS-type) display system and mea- number of clusters in the data set. Assessment of cluster validity is

surement of binocular depth perception," Jrrptrti J . Med. Elecrroti. based on performance measures using hypervolnme and density cri-

B i d . E n g . . vol. 20. no. 3. pp. 154-61, 1982.

I. J. Dowman and A. Haggag, "Digital image correlation along epi-

teria. The new algorithm is derived from a combination of the fuzzy

polar lines." in Proc. Itir. Syrnp. Iinrrge Proce.s.sing 1nrertrcriotr.s I t Y t h K-means algorithm and the fuzzy maximum likelihood estimation

Phorogruinrnerry cind Rrrnote Sensing, Gru:. , 1977, pp. 47-49. (FMLE). The UFP-ONC (unsupervised fuzzy partition-optimal num-

R. R . Real and Y . Fujiinoto, "Digital processing of dynamic imagery ber of classes) algorithm performs well in situations of large variability

for photogrammetric applications." IEEE Trutis. I n . ~ r r r ~Mcn.surc,-

~n. of cluster shapes, densities, and number of data points in each cluster.

metir. vol. IM-33. no. I . pp. 45-51. 1987. It has been tested on a number of simulated and real data sets.

M. D. Levine. D. A. O'Hanley. and G . M. Y q i . "Computer deter-

mination of depth maps." Compur. Gruphics Itrruge Proc~essing.vol. Index Terms-Clustering of sleep EEG, fuzzy clustering, hyperellip-

2 . p ~ 131-150.

. 1973. toidal clusters, performance measures for cluster validity, unequally

A. M. Chong, M . S . Beck, M. B. Brown, I. S . MacKenzie. and R. variable features, unsupervised tracking of cluster prototype.

T. Ritchings. "Micro-computer based stereoscopic rangefinding." in

Prof.. IRE Cor$ Mcinchester. 1982, pp. 95-10?,

V . P. Bennet and N . Balasubramanian, "Detection of image coinci- I. INTRODUCTION

dence using correlation coefficient measurement." i n Proc. 3Yrli ConJ

Amer. Soc. Phorogmr,ir,ierr~,Washington, DC. 1973, pp. I - 19. Cluster analysis is based o n partitioning a collection of data

J . G . Hardy and A. T. Zavodny, "Automation reconnaissance-based points into a number of subgroups, where the objects inside a clus-

target-coordinate determination ." in Pro(.. Tc~c~hriiques cirid Applicci- ter (a subgroup) show a certain degree of closeness o r similarity.

rions o f l m u g e Under.rtcrrrditig, Soc. Photo-Opt. Instrum. Eng., 1973. Hard clustering assigns each data point (feature vector) t o o n e and

pp. 95-104. only o n e of the clusters, with a degree of membership equal t o one,

S . T. Barnard and W. B. Thompson. "Disparity analysis ofirnages," assuming well defined boundaries between the clusters. This model

IEEE Truns. Ptrtrc.rtr Anul. Mudirrrr Intell. vol. PAMI-2, no. 4. pp. often does not reflect the description o f real data, where boundaries

333-340. 1980.

R . D. Arnold and T. 0. Binford, "Geometric constraints in stereo

vision." in Proc.. Iinugr Processitigfor Missilr Guidmcr. Soc. Photo- Manuscript received June 26. 1987; revised May 5, 1988. Recom-

Opt. Instrum. Eng.. vol. 238. 1980. pp. 281-290. mended for acceptance by A. K. Jain. This work was supported by the

L . Sidney. "Geometric constraints for interpreting images of coin- Kennedy-Leigh fund for Biomedical Engineering Research.

mon structural elements: Orthogonal trihedral vertices." in Proc. The authors are with the Department of Biomedical Engineering. Tech-

Techniques mid Applic~itions of Irncrge Understunding, Soc. Photo- nion, Haifa 32000, Israel.

Opt. Inatrum. Eng.. vol. 281. 1981. pp. 332-338. IEEE Log Number 8927500.

7 74 IEEE TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. 11. N O . 7. JULY 1989

between subgroups might be fuzzy, and where a more nuanced de- of data points, K is number of clusters. The parameter q is the

scription of the objects affinity to the specific cluster is required. weighting exponent for uli and controls the fuzziness of the re-

Thus, numerous problems in the life sciences are better tackled by sulting clusters [ 111.

decision making in a fuzzy environment [I]-[4]. Bezdek [5] de- Fuzzy partition is carried out through an iterative optimization

veloped a family of clustering algorithms, based on fuzzy exten- of (1) according t o [ 5 ] :

sion of the least-square error criterion, and proved the convergence 1) Choose primary centroids V , (prototypes).

of the algorithms to a local minimum [6]. Related algorithms, tak- 2) Compute the degree of membership of all feature vectors in

ing into account differences in cluster shapes have been proposed all the clusters:

by Bezdek and Dunn [7], Bezdek et al. [SI, [9], and Gustafson and I (q-ll

Kessel [ l o ] .

There are three major difficulties encountered during fuzzy clus-

tering of real data: 1) The number of clusters can not always be

defined apriori, and one has to find a cluster validity criterion [ 111,

in order to determine the optimal number of clusters present in the

data. 2) The character and location of cluster centroids is not nec-

essarily known a p r i o r i , and initial guesses have to be made. 3) 3) Compute new centroids v,:

The presence of large variability in cluster shapes, variations in

cluster densities, and variability in the number of data points in

each cluster. A good example which demonstrates the complexity

of handling real data is classification of E E G recordings [12], [ 131. (3)

Fuzzy clustering of sleep E E G in order to classify the signal into

various sleep stages is reported in [ I ] , [4], [14].

c

,=I

In the present study an algorithm for fuzzy classification into

optimal number of clusters will be described. Optimality is re- and update the degree of memberships, u j i to 6,. according to (2).

stricted here to the notion of optimizing new performance mea- 4)

sures, based on cluster hypervolume and density criteria. The al- if max [ 1 ufi - Li,, 1 ] < t stop, otherwise goto step 3 (4)

gorithm accounts for variability in cluster shapes, cluster densities, i,

and the number of data points in each of the subsets. Classification where t is a termination criterion between 0 and 1.

prototypes for initiation of the iterative process are generated

Computation of the degree of membership U , ] depends on the

through a process of unsupervised learning. The new algorithm will

definition of the distance measure, d 2 ( X , , V z ) ,[ l l ] :

be tested on different classes of simulated data, and on a real data

set derived from sleep E E G signal.

d2(X,. VI) = ( X I - &I(X, - V,). (5)

F U Z Z YPARTITION

11. UNSUPERVISED The inclusion of A ( a n rn X rn positive-definite matrix) in the dis-

In order to obtain satisfactory solution to the problem of large tance measure results in weighting according to the statistical prop-

variability in cluster shapes and densities, and to the problem of erties of the features [lo]. In the following, two different distance

unsupervised tracking of classification prototypes, a two-layer measures will be defined, to be used in the two different layers of

clustering strategy has been developed. During the first step, a the clustering process:

modification of the fuzzy K-means algorithm [5] is carried out. 1) For the case where A equals the identity matrix the distance

There are no initial conditions on the location of cluster centroids, is Euclidean. The resulting algorithm is the fuzzy K-means.

and classification prototypes are identified during a process of un- 2) For hyperellipsoidal clusters, as well as in the presence of

supervised learning. Using these prototypes, the second step in- variable cluster densities and unequal numbers of data points in

volves the utilization of a second clustering algorithm in order to each cluster, an exponential distance measure, d f ( X , , V,), based

achieve optimal fuzzy partition. This scheme is iterated for increas- on maximum likelihood estimation [7], [ l l ] , 1151 is defined. This

ing number of clusters in the data set, computing performance distance will be used in calculation of h ( i I X i ) , the posterior prob-

measures in each run, until partition into optimal number of ability (the probability of selecting the ith cluster given t h e j t h fea-

subgroups is obtained: ture vector):

1) Cluster with fuzzy K-means (Section 11-A), using unsuper-

vised tracking o f initial classification prototypes (Section 11-B).

2) Cluster with the fuzzy modification of the maximum likeli-

hood estimation (refinement of step 1. Section 11-A).

3) Compute-performance measures (Section 11-C).

4) Increase K (number of subgroups) and repeat steps 1-3 until

optimum value of performance measure is obtained.

The fuzzy K-means algorithm [ 5 ] is based on minimization of where F, is the fuzzy covariance matrix of the ith cluster, and P,,

the following objective function, with respect to U , a fuzzy K-par- the a priori probability of selecting the ith cluster.

tition of the data set, and to V , a set of K prototypes: Comparison of (6) and (2) shows that for q = 2 h ( i 1 X , ) is sim-

ilar to uli.Thus, substituting (6) instead of (2) in step 2 of the fuzzy

N k

K-means algorithm results in the fuzzy modification of the maxi-

J,(u.V ) = , =cI , c

=I

( u l i ) q d 2 ( X ,V, , ) ; K 5 N (1) mum likelihood estimation ( F M L E ) . Step 3 of the F M L E algorithm

includes, in addition to computation of the new centroid, calcula-

where q is any real number greater than I , X , is the j t h rn-dimen- tion of P , , the U priori probability of selecting the ith cluster:

sional feature vector, V, is the centroid of the ith cluster, U,/is the

degree of membership of X i in the ith cluster, d 2 ( X i , V , ) is any 1

inner product metric (distance between X , and V , ) , N is the number

P, = -

N ~ = I

C h(i (x~) (8)

IEEE TRANSACTIOXS ON PATTERN ANALYSIS A N D MACHINE INTELLICENCF. VOL I I . NO 7. J U L Y I Y X Y 175

and of F,. the fuzzy covariance matrix of the ith cluster: The performance measures were based on criteria for hypervol-

I ume and density. Fuzzy hypervolume, FHV,is defined by:

F, =

c h(i IX/)(X, - V , ) ( X ,- V J J

(9)

where F, is given by (9).

Average partition density D , is calculated from:

Due to the exponential distance function incorporated in the . K

F M L E algorithm it seeks an optimum in a narrow local region. It

therefore does not perform well. and might be even unstable during

unsupervised identification of classification prototypes described in

Section 11-C. Its major advantage is obtaining good partition results where S , , the sum of central members, is given by:

in cases of unequally variable features and densities, but only when

starting from good classification prototypes. The first layer of

the algorithm (unsupervised tracking of initial centroids) is there-

fore based on the fuzzy K-means algorithm, whereas in the next

phase optimal fuzzy partition is being carried out with the F M L E

algorithm. taking into account only those members within the hyperellipsoid,

whose radii are the standard deviations of the cluster features.

B. Unsupervised Tracking of Cluster Prototxpes The partition density Po is calculated from

The algorithms described in the previous section start with initial S

guesses of classification prototypes, and the iterative process re- P, = - (13)

FH V

sults in convergence of the cluster centroids to a local optimum. where

Different choices of classification prototypes may lead to conver-

gence to different local optima, i . e . , to different partitions. In many

practical situations a priori knowledge of the approximate loca-

tions of the initial centroids does not exist, and in order to achieve v X, { X , : ( X , - V!)F:(X, V,) < I } .

E - (14)

optimal partition unsupervised tracking of classification prototypes

is required. An example for estimating the optimal number of subsets in a

Given a partition into k clusters, the basic idea is to place the ( k data set, using the performance measures, is demonstrated in Fig.

+ 1 )st cluster center in a region where data points have low degree 1. The data set is the 150 patterns describing three iris subspecies

of membership in the existing k clusters. The following scheme [17], [18]. Plotting the performance measures FHv and PD as a

describes the steps for the selection of initial cluster centers, in- function of the number of subgroups in the data set shows points

corporated in the fuzzy K-means algorithm: of extremum at k = 3, in accordance with the botanically correct

I ) Compute average and standard deviation of the whole data number of classes.

set. The FHv criterion shows a clear extremum in most of the cases.

2 ) Choose the first initial cluster prototype at the average loca- However, the density criteria will be more sensitive as performance

tion of all feature vectors. measures when there are substantial overlapping between the clus-

3) Choose an additional classification prototype equally distant ters and when large variability in compactness of the clusters ex-

(with a given number of standard deviations) from all data points ists. The D,, criterion reflects the presence of single dense clusters

(a nonphysical location). (the fuzzy density is calculated for each cluster and then averaged

4) Calculate a new partition of the data set according to steps 1 over all clusters), and thus, partition resulting in both dense and

and 2 of the scheme outlined in Section 11. loose clusters is considered a good partition because of the dense

5 ) If k. the number of clusters is less than a given maximum, substructures. The PD criterion expresses the general partition den-

goto 3. otherwise stop. sity according to the physical definition of density.

During clustering of real data one usually has to make assump- In order to test the performance of the algorithm a simulation

tions a s to the number of underlying subgroups present in the data program was written, generating N artificial rn-dimensional feature

set. When no a priori information exists as to the internal structure vectors from a multivariate normal distribution. The input to the

of the data. or in case of conflicting evidence about the optimal program consisted o f : 1) N , the number of data points. 2) m , di-

number of subgroups, performance measures for comparison be- mension of feature space. 3) K , the number of required subsets in

tween the goodness of partitions with different numbers of clusters the data. 4) The required m-dimensional cluster prototypes. 5 ) The

need to be formulated. variance of each feature in each of the clusters. 6) The relative

A goal-directed approach [I61 to the cluster validity problem can number of data points in each subset. By choosing the distances

be chosen, where the goal is classification, in the sense of mini- between cluster prototypes to be near each other, and controlling

mization of the classification error rate. Hence, one may accept the the variance of the features, overlapping between clusters could be

basic heuristic that good clusters are actually not very fuzzy obtained, resulting in a fuzzy environment. The features had un-

[ 1 I]. Therefore, the criteria for the definition of optimal parti- equal variance generating hyperellipsoidal clusters. The number of

tion of the data into subgroups were based on three requirements: subgroups in the data, their density, and number of data points in

1) Clear separation between the resulting clusters. each subgroup were subject to variation. Another artificial data set

2 ) Minimal volume of the clusters. was taken from Gustafson and Kessel [ l o ] , and the algorithm was

3 ) Maximal number of data points concentrated in the vicinity also tested on the iris data of Anderson 1171 and Fisher [18], and

of the cluster centroid. feature vectors derived from sleep E E G . As to the algorithmic pa-

Thus, although the environment is fuzzy, the aim of the classi- rameter q , a theoretical basis for an optimal choice of the weighting

fication is generation of well-defined subgroups, and hence these exponent is so far not available [ 1 I ] , [19], [20]. A value of q = 2

requirements lead to a harder partitioning of the data set. [7], [IO], [ I l l , [21] was chosen for the U F P - O N C algorithm.

776 I E E E TRANSACTIONS O N PATTERN ANALYSIS A N D M A C H I N E I N T E L L I G E N C E . VOL. I I . NO. 7. J U L Y 1Y8Y

800

600

+oo

K

P 3 4 S 6

(a)

0.30

0.20

K

I

2 3 4 5 b

(b)

Fig. 1 . Performance criteria for the Iris data. (a) FHV-Fuzzy hypervol-

ume a5 a function of the number of subgroups in the data. ( b ) Partition

densitv as a function of number of subgroups. Extrema are seen for

k = 3.

Example I : This example demonstrates optimal partition of algorithm is shown in Fig. 4 ( d ) . All the patterns have been cor-

touching clusters with large variability in cluster densities and rectly classified.

number of data points in each cluster. In Fig. 2(a), an artificial data Example 4: The iris data set of Anderson [ 171 and Fisher [ 1 S]

set with two-dimensional feature vectors drawn from a bivariate has three subgroups, two of which are overlapping. Estimation of

normal distribution is demonstrated. O n e of the subgroups in the the optimal number of substructures in the data set (whether it is 2

data is large and loose, while the other is small and shows a much or 3) is the crucial point here. The patterns are depicted in Fig.

higher density of the data points. There is no clear border between S(a), and the F,, and PD curves in Fig. 1 . The optimal number of

the subgroups. subgroups in the data set is given by the minimum of the F H vcurve

Using the fuzzy K-means algorithm alone results in misclassifi- and the maximum of the P, curve at k = 3 clusters. Partition into

cation of boundary data points, Fig. 2(b). Peripheral data points the three clusters is shown in Fig. S(b). There are 4 misclassifica-

generated by the loose cluster will be misclassified a s belonging to tions within the 150 patterns (an error of 2 . 7 percent). Three plants

the high density cluster. Application of the U F P - O N C algorithm of Iris Versicolor have been classified a s Iris Virginica, whereas

classifies correctly all 200 data points, Fig. 2(c). only one plant of Iris Virginica has been attributed to Iris Versi-

Example 2 : This example demonstrates succcssful partition of color. All SO Iris Setosa plants have been correctly classified.

linear substructures. Fig. 3(a) demonstrates two linear clusters, Example 5: Computerized scoring of sleep EEG into various

generated from a uniform distribution by Gustafson and Kessel [IO]. stages [ 11, [4], [ 141, [22] represents a typical example of handling

The two subsets consist of two long and narrow formations, at right real data by fuzzy clustering. The patterns characterizing sleep EEG

angle to each other. The cluster centers were generated to coincide segments generate a fuzzy environment, with some traits compli-

exactly with each other. Running the U F P - O N C algorithm gives cating any process of classification:

the two clusters, Fig. 3(b), with no misclassification of any of the I ) Physiologically, there are continuous transitions between the

points. sleep stages, i.e., the subgroups are not well sepuruted.

Example 3: This example shows optimal partition of a data set 2) There is a great deal of intersubject variability of the spectral

with multiple substructures. Twelve different clusters are generated features of the various sleep stages, and thefeatures have unequal

from a multivariate normal distribution, Fig. 4(a). The feature space varictnce (large variability in cluster shapes).

is five-dimensional. There is a significant variability of shapes, 3) The number of stationary E E G segments and the variability

densities, and number of patterns in each cluster. The performance of their features differ for the various sleep stages (variability in

measures for estimating the number of subgroups in the data set cluster densities and number of data points in each cluster).

are depicted in Figs. 4(b) and (c). A minimum f o r k = 12 is clearly 4 ) Thc number of sleep stages might vary between subjects (de-

seen for the FHvcriterion, a s well as a maximum for k = 12 for pends on age, pathological conditions, etc.), i.e., the number of

the partition density criterion. The partition, running the UFP-ONC subgroups in the data is not known a priori.

I E E E TRANSACTIONS O N PATTERN ANALYSIS A N D M A C H I N E I K T E L L I C E K C t ~ . VOL. I I . NO 7. J U L Y I Y X Y Ill

139

61

I

I , I,

Two hundred data points generated from a bivariate Gaussian distribu-

tion. There are two subgroups in the data, one large and loose and the

other small and dense. (b) Partition using the fuzzy K-means algorithm.

Peripheral points generated by the loose cluster are misclassitied as be-

longing to the smaller cluster. (c) The UFP-ONC algorithm classifies

correctly all 200 data points.

Thus, even if one fixes the number of subgroups in the data set, sician is given in Fig. 6 ( e ) .There is a clear similarity between the

fuzzy clustering of sleep E E G using either of the algorithms de- two classification histograms. Due to the scanty number of EEG

scribed in 151, [IO], [19] does not guarantee optimal partition. segments belonging to sleep stage I, sleep stage wake and sleep

Patterns representing a whole nights sleep E E G segments from stage I have been classified by the U F P - O N C algorithm as being

a 30 year old female are shown in Fig. 6(a). The five features, one class.

derived by adaptive segmentation and time-dependent clustering of The C P U requirements of the U F P - O N C algorithm on the IBM

the signal [4], are the relative power in the physiological frequency AT personal computer, analyzing four-dimensional 150 patterns

bands, delta, theta, alpha, sigma. and beta. One of the criteria for (iris data), with K , the maximal number of clusters equal to 6, was

estimating the optimal number of classes in the data FIrv is plotted 14 min.

in Fig. 6(b). From the minimum in the curve it can be concluded

that there are five subgroups. T h e partition and classification his- IV. CONCLUSIONS

togram (hypnogram) are depicted in Figs. 6(c) and (d). respec- Implementing the strategy of unsupervised tracking of initial

tively. For comparison, a hypnogram scored manually by a phy- cluster centroids, the most flexible algorithm has been found to be

778 I E E E TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. I I . NO 7. JULY 198)

43

.* ..

0 20

0

0 20

j

Fig. 3 . Partition of linear clusters, data of Gustafson and Kessel [ I O ] . (a)

Twenty data points drawn from a uniform distribution. The two subsets

consist of two long and narrow formations, at right angle to each other.

(b) Convergence of the two centroids to their final locations running the

UFP-ONC algorithm. The trajectories of the two centroids during the

iterations can be followed by the points denoted by small numerals 1 and

2. Two standard deviations are drawn around the final centroids. All data

points have been correctly classified.

I I

+ 8 12 16 20

99

60

4 B 12 lb ZU

variate Gaussian distribution with unequally variable features. variable

densities and variable number of data points in each cluster. (a) Data

points before partition. Only three of the features are displayed. (b), (c)

Fuzzy hypervolume (FHV) and partition density, as a function of the

number of subgroups in the data. Extrema for k = 12 can be seen. (d)

Partition of 12 subgroups using the UFP-ONC algorithm. All data points

have been correctly classified.

the fuzzy K-means, although it does not give optimal partition in count, the F M L E algorithm is superior to Gustafson and Kessels

cases of variable cluster shapes and densities. O n the other hand, [lo] fuzzy covariance algorithm, in that it does not require an extra

using an exponential distance measure including the fuzzy co- volume constraint (the p, of Gustafson and Kessel), limitation on

variance matrix (the F M L E algorithm [7], [ I l ] , [15]) results in the hypervolume being achieved through the exponent.

optimal partition even when a great variability of cluster shapes The new algorithm described in the present study combines the

and densities is present. A n optimal performance of the F M L E al- favorable features of both the fuzzy K-means algorithm and the

gorithm requires starting from good seed points, because due to F M L E , together with unsupervised tracking of classification pro-

the exponential distance this algorithm converges to a local op- totypes. Optimal partition has been achieved with the U F P - O N C

timum in a rather narrow region. Taking this limitation into ac- algorithm for several synthetic data sets, as well as for sleep E E G

1EF.F. TRANSACTIONS ON PATTERN ANALYSIS A N D M A C H I N E INTELLIGENCk. VOL 1 1 . NO. 7. JULY 1989 779

Fig. 5. Classificat~onof the Iris data of Anderson and Fisher using the

UFP-ONC algorithm. (a) 150 four-dimensional feature vectors. Only

three of the features are displayed here: PL-Petal length. PW-Petal

width. SL-Sepal length. (b) Fuzzy partition to three subgroups. There

are a total of four errors, three items of Iris Versicolor have been mis-

classified as Iris Virginica whereas one Iris Virginica has been misclas-

sified as Iris Versicolor.

classification, omitting the need for initial guesses on cluster pro- and Moreau (301 developed a method for cluster validity based on

totypes. The iris data set [17], [I81 is a well known example of the bootstrap technique, that could be used with any clustering al-

overlapping substructures. The results of applying the UFP-ONC gorithm. Using a criterion based on Davis and Bouldins [29] clus-

algorithm in this case were optimal. both from the point of view of ter separation measure, and on cluster compactness measure

estimating the number of underlying substructures, and that of clas- (within-cluster scatter), both the K-means and Ward clustering al-

sification error rate [23]-[25]. Extending the notion of hyperellip- gorithms succeeded in detecting the botanically correct number of

soidal clusters to the extreme, by letting one feature vary much less classes for the iris data set.

than the others. gives rise to line-like clusters. Graph-theoretic

methods have been proven t o be successful in detecting linear sub- ACKNOWLEDGMENT

structures 1261, but in general these methods fail on hyperellip-

The authors wish to thank Dr. E. Bar-On for valuable discus-

soidal clusters [8]. Due to the inclusion of the F M L E in the UFP-

sions, and to Prof. A . K. Jain for reading the manuscript and pro-

ONC algorithm. the new algorithm is also able to detect line-like

viding many useful suggestions. The research was supported by the

clusters, a s demonstrated on the data of Gustafson and Kessel [ 101.

Kennedy-Leigh fund for Biomedical Engineering Research.

Performance measures for assessing cluster validity have been

proposed in the framework of ranking various partitions obtained

REFERENCES

from different clustering algorithms. Such a cluster validity strat-

egy was implemented in [16], using performance measures based [ 11 L. Larsen, E . Ruspini. J . McDew, D. Walter. and W. Adey. A test

on fuzzy decomposition of the data. The search for a proper cluster of sleep staging system in the unrestrained chimpanzee, Brain Res.,

validity criterion in the present study has been goal-oriented, with vol. 40, pp. 319-343, 1972.

relation to the application domain [16], [27]. The aim was to es- 121 J . C. Bezdek, Feature selection for binary data: Medical diagnosis

timate the optimal number of substructures in the data set for the with fuzzy sets. in Proc. 25th Nut. Computer Conf., 1976, pp. 1057-

purpose of classification (minimum classification error rate). It has 1068.

[ 3 ] J . C. Bezdek and W . A. Fordon, Analysis of hypertensive patients

been motivated by studies of automatic classification of sleep stages by the use of fuzzy ISODATA algorithm. in Proc. JACC, vol. 3 ,

141. [14], [22], where the number of subsets in the data is not nec- 1978, pp. 349-356.

essarily known a p r i o r i , and where a large intersubject variability 14) I . Gath and E. Bar-On, Computerized method for scoring of poly-

of the number of classes may be present. graphic sleep recordings, Cornput. Progr. Biomed., vol. 1 1 , pp.

In order to estimate the optimal number of subgroups present in 217-223, 1980.

the data the U F P - O N C algorithm incorporates performance mca- [ 5 ] J . C. Bezdek, Fuzzy mathematics in pattern classification. Ph.D.

sures based on hypervolume and density criteria. The hypervolume dissertation, Cornell Univ.. Ithaca. NY. 1973.

criterion is related to the within-cluster scatter, but due to its fuzzy [6] -, A convergence theorem for the fuzzy ISODATA clustering

characteristics the FHv,unlike the square error criterion, is not a algorithms, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-

2, no. I , pp. 1-8, 1980.

monotone function of k . These performance measures (and in par- [7] J . C. Bezdek and J . C. Dunn. Optimal fuzzy partition: A heuristic

ticular the hypervolume criterion) plotted as a function of the num- for estimating the parameters in a mixture of normal distributions,

ber of clusters k show a clear extremum. from which conclusions IEEE Trans. Cornput.. vol. C-24, pp. 835-838, 1975.

as to the optimal number of substructures in the data can be drawn. 181 J . C. Bezdek. C. Coray, R . Gunderson, and J. Watson, Detection

This has been demonstrated for the iris data, where the botanically and characterization of cluster substructure. 1. Linear structure: Fuzzy

correct number of clusters was detected by the new algorithm. c-lines, S I A M J . Appl. Math., vol. 40, pp. 339-357, 1981.

Other performance measures, aimed at delineating the number [9] J . J. C. Bezdek, C . Coray. R. Gunderson, and J. Watson, Detection

of subgroups in the data set, are either monotone functions of k and characterization of cluster substructure. 11. Fuzzy c-varieties and

[ l l ] . [28], or show a very slight preference for a certain value of convex combinations thereof, SIAM J . Appl. Math.. vol. 40, pp.

358-372. 1981.

k . as is the case with Windhams proportional exponent and the [IO] E. E. Gustafson and W. C. Kessel, Fuzzy clustering with a fuzzy

U D F criterion [24], [21] applied to the iris data. The cluster sep- covariance matrix, in Proc. IEEE CDC, San Diego, CA, 1979, pp.

aration measure of Davis and Bouldin 1291 failed to uncover the 76 1-766.

botanically correct number of classes for the iris data, in addition [ 1 1 1 J . C. Bezdek. Pattern Recognition with Fuzzy Objective Function Al-

to exhibiting two extra local minima, botanically meaningless. Jain gorithms. New York: Plenum, 1981.

780 IEEE T R A N S A C T I O N S ON PATTEKN ANALYSIS A N D M A C H I N E I N T E L L I G k N C E . VOL I I . NO 7. J U L Y 1989

I

ia)

I K I

3 4 5 6 7

segmentation of a whole nights sleep EEG.The five-dimensional feature

vectors include the relative power in the physiological frequency bands

delta, theta. alpha. beta. and sigma. (a) Data points before partition. (b)

Performance measures. Fuzzy hypervolume as a function of k , the n u n -

her of subgroups in the data. A minimum for k = 5 can be seen. (c)

Partition using the UFP-ONC algorithm. (d) Classification histogram.

(a)-(e) are the various classes. (e) Manual scoring of the same EEG as

in (a)-(d) by a physician into sleep stages. W-waking. REM-rapid eye

movement sleep. I. 11. 111. IV-non-REM sleep stages.

IEEE T R A N S A C T I O N S ON PA-rTERK ANALYSIS A N D M A C H I N E INTELLIGb.NCE. V O L . I I . NO. 7. JLJLY 1989 78 I

1171 A. S . Gevins. Pattern recognition o f human brain electrical poten- [21) M . P. Windham. Cluster validity for the fuzzy c-means clustering

tials. IEEE Truris. Putterri Amrl. Machirie Intell.. vol. PAMI-2, no. algorithm, IEEE Traris. Puttern Anul. Machine Intell., vol. PAMI-

5 . pp. 383-404. 1980. 4 , n o . 4 , pp. 357-363, 1982.

[13] B. H. Jansen and W .K . Cheng. Classification of sleep patterns by [ 2 2 ] I . Gath and E. Bar-On. Sequential fuzzy clustering of sleep EEG

means of markov modeling and correspondence analysis. IEEE recordings, in Methods of Sleep Research, S. Kubicki and W .

Truns. Putterri A i i d . Machirie fritell.. vol. PAMI-9, no. 5, pp. 707- Herrmann, Eds. Jena, Germany: Gustav Fischer, 1985. pp. 55-64.

710. 1987. 1231 J . C. Bezdek, Numerical taxonomy with fuzzy sets, J . M u r k Biol.,

1141 I. Gath and E. Bar-On. Classical sleep stages and the spectral con- vol. 1 - 1 , pp. 57-71, 1974.

tent of the EEG signal. fnr. J . Neurosci.. vol. 22. pp. 147-155, [24] M. P. Windham, Cluster validity for fuzzy clustering algorithms.

1983. F u z y Spt.5 Sysr., vol. 3, pp. 1-9, 1980.

[ I S ] N . E.Day. Estimating the components of a mixture of normal dis- [ 2 5 ] T. Gou and B. Dubuisson, A loose-pattern process approach to clus-

tributions. Biornetrika, vol. 56, pp. 463-474, 1969. tering fuzzy data sets. IEEE Trans. Parterri Aria/. Machine. f n t r l l . ,

[ 161 E.Backer and A. K. Jain. A clustering performance measure based vol. PAMI-7. no. 3 , pp. 366-372, 1985.

on fuzzy set decomposition. IEEE Trans. Parrern Aiiul. Muchiric, 1261 C. T. Zahn. Graph-theoretical methods for detecting and describing

Intell.. vol. PAMI-3. no. 1 . pp. 66-74. 1981. gestalt clusters, fEEE Trcriis. Cornput.. vol. C-20, no. I , pp. 68-

[ 171 E.Anderson, The irises of the Gaspe peninsula, Bull. Anier. Iris 86, 1971.

Soc.. vol. 59, pp. 2-5. 1935. 1271 R . Dubes and A. K. Jain, Validity studies in clustering methodol-

[ 181 R . A. Fisher, The use of multiple measurements in taxonomic prob- ogies, Puttrnr Recognition, vol. I I . pp. 235-254, 1979.

lems. Ann. Eugeriics. vol. 7. pp. 179-188. 1936. 1281 P. H. A. Sneath and R . Sokal. Nurnericul Taxonorny. San Francisco,

1191 K. Leszczynski. P. Penczek. and W . Grochulski. Sugenos fuzzy CA: Freeman, 1973.

measure and fuzzy clustering. Fuzzy Sets Sysr.. vol. 15, pp. 147- 1291 D. L. Davis and D. W . Bouldln. A cluster separation measure,

158. 1985. IEEE Trans. Pattern A m / . Machirie fntrll.. vol. PAMI-1, no. 2, pp.

(201 R . L. Cannon, J . V. Dave. and J . C. Bezdek, Efficient implemen- 224-227, 1979.

tation of the fuzzy c-means clustering algorithms. IEEE Traris. Pat- 1301 A. K. Jain and J. V . Moreau, Bootstrap technique in cluster anal-

terri Ancrl. Machine Intell.. vol. PAMI-8, no. 2, pp, 248-255. 1986. ysis, Pattern Recognition, vol. 20, no. 5 , pp. 547-568, 1987.

- A Survey of Methodaology of Fraud Detection Using Data MiningTransféré parEditor IJTSRD
- Clustering PresentationTransféré parCarlos Ademir
- Cluster Analysis 1Transféré parocmain
- 541-693-1-PBTransféré parjeffa123
- Applying DM in MLTransféré parİnönüa
- DIP Final Project ReportTransféré parRajkumar Kondappan
- Enrollments Forecasting Based On Aggregated K-Means Clustering and Fuzzy Time SeriesTransféré parIRJET Journal
- Advance Quantitiative ThecniqueTransféré parfahimafridi
- IJETTCS-2013-10-25-084Transféré parAnonymous vQrJlEN
- Report.docxTransféré paryumnabilal
- 183800Transféré parapi-19739582
- M.TechTransféré parAli
- Paper 28 - Clustering as a Data Mining Technique in Health Hazards of High Levels of Fluoride in Potable WaterTransféré parEditor IJACSA
- Fuzzy clustering Approach in segmentation of T1-T2 brain MRITransféré parIDES
- Data Mining Practice Final SolTransféré parminichel
- 10.1137@1.9780898718348.ch10Transféré parMauro Avila
- Feature Subset Selection Techniques - Swift Clustering and Principle Component AnalysisTransféré parseventhsensegroup
- 2016 IEEE ns2 Projects| Dynamic Cluster Head Selection Method for Wireless Sensor NetworkTransféré parLakshmiDhanam
- adhocTransféré parrajouta19
- daweqwTransféré parChillik Ardianto
- Generating Diverse and Representative Image Search Results for LandmarksTransféré parmachinelearner
- cpmpapTransféré parPraveen Kumar G
- Active Clustering: Robust and Efficient Hierarchical Clustering using Adaptively Selected SimilaritiesTransféré parAyayitoxCraft
- Cc 31331335Transféré parIJMER
- EDGE DETECTION OF MICROSCOPIC IMAGETransféré parIAEME Publication
- Report Fin 2Transféré parvgmanjunatha
- SOR15-eGovernment-ver 7.docxTransféré parMirjana Pejic Bach
- Machine LearningTransféré pardarebusi1
- DMQBTransféré parRahul Dhande
- 2 .Data Mining Tasks.pptTransféré parshreya

- 6Transféré parKranthirekha Chennaboina
- 2874V Hover Bike Tech YESTransféré parKranthirekha Chennaboina
- Gabor Texture SegmentTransféré parKranthirekha Chennaboina
- another paper.pdfTransféré parKranthirekha Chennaboina
- 1609.02374Transféré parKranthirekha Chennaboina
- Techexpo 19 ResponsibilitiesTransféré parKranthirekha Chennaboina
- igarss2015Transféré parKranthirekha Chennaboina
- Dip MidpaperTransféré parKranthirekha Chennaboina
- EECE 301 Note Set 1 OverviewTransféré parrodriguesvasco
- My PaperTransféré parKranthirekha Chennaboina
- EE370 LabManualTransféré parimwaqas
- EE3002 Past Exam PaperTransféré parKranthirekha Chennaboina
- EE3002 Past Exam PaperTransféré parKranthirekha Chennaboina
- Assignment 4Transféré parKranthirekha Chennaboina
- Notes 01Transféré parKranthirekha Chennaboina
- Day 2 Class NotesTransféré parKranthirekha Chennaboina
- dspTransféré parKranthirekha Chennaboina

- OS6850Transféré parDzung Chi Dang
- DGND3300v2_UM_14Oct10Transféré parDPL_1971
- AS400 intTransféré parHotPriyu Onlyforgals
- SEPT 12207 2008 Checklist-AC1 Sample PagesTransféré parRey2000
- Testing SandboxTransféré parqweqwqwe
- Lab Windows User GuideTransféré parBala Venugopal
- Basic Level Manual - Version 2.8Transféré parhorns2034
- L5-GamesTransféré parUsama Chaudhary
- Project Management q & AnsTransféré pargotu123
- Bash Cookbook - Ron BrashTransféré parisampalis
- Balloon ShootTransféré parDarshankirik
- ERP Failure at HersheysTransféré parBhargav Singirikonda
- Abacus Intro11Transféré parFelipe Pérez Pérez
- 4008 InstallTransféré parJM CE
- handset_to_sipTransféré parapi-19663123
- 800-12625-E_Rapid_Eye_LT_Test_Procedures[1].pdfTransféré parJeovany Batista
- xi4_ds_perf_opt_enTransféré parSubhendu Swain
- Easy ListTransféré parLuis Cabello
- RU50 Feature Descriptions of NSNTransféré parmanuel
- sony vpc-eaTransféré parBanyugea
- Docu54199 OneFS 7.1.1 CLI Administration GuideTransféré partimeprati
- DMarket White Paper EnTransféré parMike Paniwan
- Section 3Transféré parAbdul Ghani Khan
- 3710709Transféré parBhavik Prajapati
- PO lsmw reqTransféré parKishore Kumar
- S2A9900 Data Sheet 5Transféré parJeff Baker
- HISTORY OF NORDIC COMPUTINGTransféré parNihal
- ZBrush4 ZAppLink DocumentationTransféré parYouri Fa
- New Fuzzy Logic Based Intrusion Detection SystemTransféré parInternational Journal for Scientific Research and Development - IJSRD
- Vignette Dialog Product DatasheetTransféré parvignettecorp