Vous êtes sur la page 1sur 10

10.1098/rspa.2000.

0650

Mean, median and mode ltering of images


By L e w i s D. G rif fi n
Medical Imaging Science Interdisciplinary Research Group,
Kings College, London, UK (lewis.gri n@kcl.ac.uk)

Received 29 March 2000; revised 17 July 2000; accepted 8 August 2000

If a two-dimensional image is simpli ed by repeatedly replacing its values with the


mean in an in nitesimal neighbourhood, it evolves according to the diusion equation
Lt = Lxx + Lyy (subscripts denote dierentiation). This equation can alternatively
be written in gauge coordinates as Lt = Lvv + Lww , where the v-direction is tangent
to the isophote and the w-direction is along the gradient. The alternative evolution
scheme of Lt = Lvv has also attracted attention. Guichard & Morel showed, in 1996,
that this equation describes the operation of repeated in nitesimal median ltering.
In this paper it is proved that repeated in nitesimal mode ltering is described by
Lt = Lvv 2Lww at regular points, and Lt = 0 at critical points. Other new results
are (i) an approximate equation for median ltering at critical points, and (ii) a
derivation of the equation for median ltering at regular points, which generalizes
to mean and mode ltering. Finally, the results of numerical implementation of all
three ltering schemes are briey presented.
Keywords: scale space; anisotropic di usion; measures of central location;
partial di erential equations; mean curvature ow

1. Introduction
The concept of progressively simplifying an image is well established (Marr 1982; ter
Haar Romeny 1994; Gri n 1995). The obvious technique is to replace simultaneously
the value at each point of the image by the average of the values within an aper-
ture around the point. Iterating this procedure progressively simpli es the image.
The eect of such ltering will depend upon the size and shape of the aperture.
To remove this dependency it is natural to consider the limiting process as smaller
apertures are used and the number of iterations is increased. In the limit, the aper-
tures become in nitesimal and the iteration number is replaced by a continuous time
parameter. At the limit, the eect of ltering can only be dependent on dierential
measures of the image, and so in nitesimal ltering is always describable by a par-
tial dierential equation (PDE). PDE schemes for processing images are becoming
increasingly important (Weickert 1998).
Dierent de nitions of average result in dierent ltering schemes (Davies 1988;
Evans & Nixon 1995; Torroba et al . 1994). Using the mean as the averaging operator
results in a ltering scheme equivalent to linear diusion (Koenderink 1984). Linear
diusion is described (in the case of two-dimensional images) by the equation Lt2 =
Lxx + Lyy , where x and y are image coordinates and t2 is the time parameter.
Given that mean ltering is described by a simple equation and has found wide
applicability in image processing, it seems productive to consider the eect of using

Proc. R. Soc. Lond. A (2000) 456, 2995{3004 c 2000 The Royal Society
2995
2996 L. D. Gri n

Figure 1. The gauge coordinate systems used in the paper. The w; v-gauge is used at regular
points: the w-direction is along the gradient, and the v-direction is tangent to the isophote.
The p; q-gauge is used at critical points (extremum and saddles); it is chosen such that the
mixed derivative, Lp q , disappears; and the magnitude of the second derivative in the p-direction
exceeds that in the q-direction.

averaging operators other than the mean, in particular the median and mode. The
case of median ltering has been analysed by Guichard & Morel (1995), who showed
that in the limiting case, at points with non-zero gradient (regular points), the image
evolves according to
L2y Lxx 2Lx Ly Lxy + L2x Lyy
Lt1 = :
L2x + L2y
This equation may be written more simply in gauge coordinates. In particular,
in w; v-gauge coordinates, where the w-direction is along the gradient and the v-
direction is tangent to the isophote ( gure 1a), the equation becomes Lt1 = Lvv .
Mean ltering is described, in such coordinates, by Lt2 = Lvv + Lww .
The main result proved in this paper (conjectured in Gri n (1998)) is that mode
ltering is described, at regular points, by Lt0 = Lvv 2Lww . Exact and approxi-
mate equations for mode and median ltering, respectively, at critical points are also
derived. Finally, results of numerical implementations of mean, median and mode
ltering are briey presented.

2. Mathematical preliminaries
The equivalence between mean ltering and linear diusion is almost independent
of the shape of the apertures considered when deriving the action of mean ltering.
This is because
(i) mean ltering is equivalent to convolution with the lter aperture;
(ii) n-times application of a convolution kernel is equal to the single application of
the n-times convolution product of the kernel; and
(iii) the limiting n-times product of most symmetrical kernels is the same, a two-
dimensional Gaussian.
Given (iii) (which is the central limit theorem), the most natural apertures to consider
are Gaussian apertures of non-zero width. Additional arguments for the naturalness
of Gaussian apertures have been made (Koenderink 1984; Alvarez et al . 1993).

Proc. R. Soc. Lond. A (2000)


Filtering of images 2997

Figure 2. Example image regions (left column, density plot) centred on regular and critical
points. The central column shows a Gaussian aperture. The right column shows the histograms
of the images on the left within the aperture shown in the centre. The mean, median and mode
of these histograms are marked, as are the corresponding isophotes in the underlying images
(left). The median isophotes divide the image into two regions with equal integral of the aperture
weighting.

To de ne Gaussian apertures I rst de ne the one-dimensional Gaussian function


with scale parameter t:
1 x2 =4t
Gt (x) := p e :
4 t
A two-dimensional origin-centred aperture may then be de ned as

x
Gt := Gt (x)Gt (y):
y

Proc. R. Soc. Lond. A (2000)


2998 L. D. Gri n

The eect of ltering an image L : R2 ! R with non-in nitesimal apertures is


most easily de ned in two steps (see Gri n (1997) and Koenderink & van Doorn
(1999); see also gure 2). The rst step is to form the histogram of the image values
visible within the aperture; at the origin this is given by

Ht[L](p) := Gt (r) (L(r) p);


r 2 R2

where p ranges over image values and is the Dirac delta function (Richards & Youn
1990). The second step is to operate on the histogram to produce an average value.
The three averaging operators considered here are

(i) the mean operator,

2 (D) := pD(p) D(p)


p2 R p2 R

(where D is a histogram);

(ii) the median operator,


m 1
1
1 (D) := m s.t. D(p) dp = D(p) dp;
1 2 1

(iii) and the mode operator

0 (D) := m s.t. D 0 (m) = 0; D 00 (m) < 0:

I denote the combined eect of the two steps, when performed at the origin, by
ar (t) := r (Ht[L]), where r = 0; 1; 2. Here, as elsewhere, the subscripts 0, 1 and 2
are used to indicate mode, median and mean schemes, respectively. This labelling
is not arbitrary and follows from the de nition of central locations as values where
jpjr D(p) is a local minimum (Frechet 1948).
It will be necessary to consider 0r the derivatives of the averaging operators;
de ned by
0r [D](p) := lim (1= )( r (D + p ) r (D));
! 0

where p stands for a delta function at image value p. The derivative of the mean
operator is easily shown to be 02[D](p) = p, which is independent of D because of
the linearity of the mean operator. In contrast, the nonlinearity of the median and
mode operators means that their derivatives do depend on D, so calculation will be
deferred until later sections when relevant histograms have been identi ed.
The action of ltering with in nitesimal apertures is equal to

lim (1=t)ar (t) = a0r (0):


t! 0

To calculate this, I start from the easy result that

ar (t) = t1=2 r (H 1 [t
1=2
L(t1=2 )]):

Proc. R. Soc. Lond. A (2000)


Filtering of images 2999

Taking the derivative with respect to t, one obtains

a0r (t) = 12 t 1=2 r (H1 [t


1=2
L(t1=2 )])
@ 0
+ t1=2 H1 [t 1=2 L(t1=2 )](p) r [H 1 [t
1=2
L(t1=2 )]](p): (2.1)
p2 R @t
The expression in (2.1) within curly braces can be expanded as follows:

@
t1=2 H1 [t 1=2 L(t1=2 )](p)
@t
= 12 t1=2 G1 (r) 0 (t 1=2 L(t1=2 r) p)( t 3=2 L(t1=2 r) + t 1 r L0 (t1=2 r)):
r 2 R2
(2.2)
Further steps in deriving the action of mean, median and mode ltering depend on
whether or not the image is regular at the point being considered, and will be taken
in the following two sections.

3. Filtering at regular points


At a regular point one may always choose orthogonal directions w and v such that
the image gradient vanishes along the v-direction and is positive in the w-direction
( gure 1, left). Thus, since addition of a constant image has a trivial eect on all
ltering schemes, the image in the neighbourhood of the origin can be described by
w
L = wLw + 12 w 2 Lww + wvLwv + 12 v 2 Lvv + ;
v
where Lw > 0. Given that one is at a regular point, the derivation started in (2.2)
can be continued and then evaluated at t = 0:
@
t1=2 H1 [t 1=2 L(t1=2 )](p)
@t t= 0
1
= G1 (w; v) 0 (wLw p)( 12 w 2 Lww + wvLwv + 12 v 2 Lvv )
2 w;v2 R
1
= (2L2w (Lvv 2Lww ) + p2 Lww )G0L2 (p): (3.1)
4L2w w

It is easily con rmed that, with appropriate scaling, the histogram of image values
within the aperture tends to Gaussian form as the aperture shrinks to a point, and
the width of the distribution is a function of the gradient, Lw , at the aperture centre.
Symbolically,
lim H1 [t 1=2 L(t1=2 )] = GL2w : (3.2)
t! 0

Equations (3.1) and (3.2) can now be used to evaluate equation (2.1) at t = 0:
1
a0r (0) = 12 a0r (0) 0 0 2
r [GL2w ](p) GL2w (p) (2Lw (Lvv 2Lww ) + p2 Lww ):
4L2w p2 R

Proc. R. Soc. Lond. A (2000)


3000 L. D. Gri n

Hence
1
a0r (0) = 0 0 2
r [GL2w ](p) GL2w (p) (2Lw (Lvv 2Lww ) + p2 Lww ): (3.3)
2L2w p2 R

Knowing that the limiting shape of the histogram is Gaussian, the appropriate deriva-
tives of the averaging operators r may now be calculated. They are
0
2[ ](p) = p; as already noted;
0
= p
1 [G L2w ] ; where is the unit step function := ;
2 Lw
Lw
00[GL2w ] = p 0 :

Inserting these into (3.3) and evaluating leads to
a02 (0) = Lvv + Lww ; a01 (0) = Lvv ; a00 (0) = Lvv 2Lww ;
and thus the PDEs for mean, median and mode ltering, at regular points, are:
Lt2 = Lvv + Lww ; mean ltering;
Lt1 = Lvv ; median ltering;
Lt0 = Lvv 2Lww ; mode ltering:

4. Filtering at critical points


At a critical point one may always choose orthogonal directions p and q such that
the mixed second derivative Lpq vanishes at the point. To resolve the ambiguity as
to which direction is p and which is q, the constraint jLpp j > jLqq j is used. Examples
of such a gauge are shown in gure 1 (centre and right). Using this gauge, the image
in the neighbourhood of a critical point is described by
p
L = 12 p2 Lpp + 12 q 2 Lqq + :
q
Taking the limit to in nitesimal apertures is easier at critical points than at regular
ones, because the irrelevant rst-order structure does not need to be nessed by
algebraic manipulation. In particular, it can be shown that
1 2
lim (1=t) r (H t [ 2 p Lpp + 12 q 2 Lqq + ]) = 1 2
r (H 1 [ 2 p Lpp + 12 q 2 Lqq ]):
t! 0

(a) Mean ltering


Mean ltering does not show any special behaviour at critical points. The diusion
equation can be written as
Lt2 = Lpp + Lqq = Lxx + Lyy = r2 L;
where r2 is the coordinate-independent Laplacian operator. This is well de ned at
critical points just as it is at regular ones.

Proc. R. Soc. Lond. A (2000)


Filtering of images 3001

(b) Mode ltering


Mode ltering does not aect the image value at critical points, i.e. Lt0 = 0. This is
apparent from considering the forms of the local histograms at such points ( gure 2,
middle and bottom). For example, at a minimum (Lpp , Lqq > 0) the histogram is
given by

H1[ 12 p2 Lpp + 12 q 2 Lqq ](v)



1 expf v=[2(cos( )2 Lpp + sin( )2 Lqq )]g
d if v > 0;
= 4 cos( )2 Lpp + sin( )2 Lqq (4.1)
0 if v < 0:

By taking the derivative of (4.1) with respect to v, the histogram at a minimum can
be shown to be decreasing for positive v and, thus, has a single maximum at v = 0.
An equation similar to (4.1) can be deduced for the histogram at a saddle point; and
again by taking derivatives it can be shown that the only maximum is at v = 0.

(c) Median ltering


Calculating the eect of median ltering at critical points is easy for the following
special cases:
Lpp = Lqq ) Lt1 = 0; a balanced saddle (Gri n & Colchester 1995);
Lqq = 0 ) Lt1 = 2Lpp erf 1 ( 12 )2 0:455(Lpp + Lqq );
eectively a one-dimensional extremum;
Lpp = Lqq ) Lt1 = 2 ln(2)Lpp 0:693(Lpp + Lqq ); a circular extremum:
To derive an exact result for median ltering at general critical points it is necessary
to integrate equation (4.1) with respect to v between limits 0 and c; and then nd
the value of c that produces an integral of value 12 . I have failed to nd an exact
solution to this procedure. Instead, by numerical methods, I have tted the following
approximate equation, which
(i) satis es the property
1 2
1 (H1 [ 2 p ( Lpp ) + 12 q 2 ( Lqq )]) = 1 2
1 (H1 [ 2 p Lpp + 12 q 2 Lqq ]);

(ii) is exact for the three special cases above; and


(iii) is accurate to within 5% for other cases:
1 2 + 12 q 2 Lqq ])
1 (H1 [ 2 p Lpp (Lpp ; Lqq )
2
Lqq Lqq
= Lpp 0:455 + 0:907 + 0:457
Lpp Lpp
3 4
Lqq Lqq
0:214 0:218 : (4.2)
Lpp Lpp

Proc. R. Soc. Lond. A (2000)


3002 L. D. Gri n

Figure 3. The e ect of mean (left column), median (centre) and mode (right) ltering. The rst
row shows the original image; the other rows reading downwards show the progressive e ect of
repeated ltering after 4, 16 and 64 iterations. The mode-ltered image at bottom right is the
nal state for this image; further mode ltering has no e ect.

5. Numerical implementation
Figure 3 shows the results of mean, median and mode ltering applied to an image
of a human eye. Full details of the algorithm will be given in a further publication. In
brief, Eulers method (Press et al . 1992) is used to compute each ltered image from
the preceding one. Spatial derivatives are calculated with respect to the weighted
neighbourhood
1 4 1
1
4 16 4
36
1 4 1
commonly used in image processing (Lindeberg 1990).
Of the three ltering methods, mean and median ltering are the most similar
in eect. The dierence between the two is most easily grasped when viewing an
animation of the ltering process. It is then clear that in mean ltering, light and dark

Proc. R. Soc. Lond. A (2000)


Filtering of images 3003

blobs spread, thin and overlap without interacting (because of the linearity of mean
ltering). In contrast, median ltering is clearly nonlinear. The impression when
viewing it is of the isophotes of the image simplifying and shrinking to points. This is
a correct impression because isophote curvature can be expressed as is o = Lvv =Lw
(ter Haar Romeny et al . 1994), and, hence, at regular points, median ltering moves
isophotes in a locally normal direction at a speed proportional to their local curvature
(Guichard & Morel 1995).
Mode ltering is distinctly dierent in eect from mean or median. As mode l-
tering progresses, the 2Lww term has the eect of de-blurring and so enhancing
edges, while the +Lvv term stabilizes this process and prevents the developing loci
of discontinuity from becoming too ragged. Away from developing edges, the image
changes in value towards nearby critical points. The nal result seems to be a mosaic
of plateaus separated by discontinuities, though this has not been proved. At this
point, the image is unaected by further applications of the mode- ltering procedure.
Inspecting the nal mode- ltered image in gure 3 one can see several features
that seem artefactual, for example the dark spur extending from the top left of the
iris region. This is con rmed when viewing the animation, when occasional `leakages
can be observed. These artefacts seem to be a consequence of the way the PDE has
been discretized rather than a genuine eect of mode ltering.

6. Concluding remarks
The new results in this paper are
(i) the PDEs for mode ltering at regular and critical points;

(ii) an approximate PDE for median ltering at critical points; and

(iii) a new proof of the PDE for median ltering at regular points which generalizes
to mean and mode ltering.
All results have been con rmed by numerical calculations (using Monte Carlo tech-
niques).
Mode ltering appears to have the potential to be a useful tool in image processing.
The aim of nding PDEs that will `clean up images and generate natural segmenta-
tions has led several authors (Gabor 1965; see also Lindenbaum et al. 1994; Kramer
& Bruckner 1975; Osher & Rudin 1990; Pollak et al. 1997) to propose similar PDEs,
primarily on ad hoc grounds.
Finally, there is an interesting relationship that exists between the PDEs for mean,
median and mode ltering at regular points, namely 3 t1 = t0 + 2 t2 . This exact
equation reminds us of the approximate relationship 3 1 0 + 2 2 that has been
observed to be true for moderately skewed histograms (Pearson 1895). That the
relationship holds exactly in this case is presumably due to the in nitesimal nature
of the histograms involved.

References
Alvarez, L., Guichard, F., Lions, P. L. & Morel, J.-M. 1993 Axioms and fundamental equations
of image processing. Arch. Ration. Mech. Analysis 123, 199{257.

Proc. R. Soc. Lond. A (2000)


3004 L. D. Gri n

Davies, E. R. 1988 On the noise suppression and image enhancement characteristics of the
median, truncated median and mode lters. Pattern Recogn. Lett. 7, 87{97.
Evans, A. N. & Nixon, M. S. 1995 Mode ltering to reduce ultrasound speckle for feature
extraction. IEE Proc. Vis. Image Sig. Process. 142, 87{94.
Fr echet, M. 1948 Les elements al eatoires de nature quelconque dans un espace distanci e. Ann.
lInstitut Henri Poincar e X, 215{308.
Gabor, D. 1965 Information theory in electron microscopy. Lab. Invest. 14, 801{807.
Gri n, L. D. 1995 Descriptions of image structure. PhD thesis, University of London.
Gri n, L. D. 1997 Scale-imprecision space. Image Vis. Comput. 15, 369{398.
Gri n, L. D. 1998 Simplifying images. Perception 27, 9.
Gri n, L. D. & Colchester, A. C. F. 1995 Supercial and deep structure in linear di usion scale
space: critical points, isophotes and separatrices. Image Vis. Comput. 13, 543{557.
Guichard, F. & Morel, J.-M. 1995 Partial di erential equations and image iterative ltering.
Preprint no. 9535, CEREMADE, Universit e Paris IX-Dauphine, 75775 Paris Cedex 16, France.
Koenderink, J. J. 1984 The structure of images. Biol. Cybern. 50, 363{370.
Koenderink, J. J. & van Doorn, A. J. 1999 The structure of locally orderless images. Int. J.
Comp. Vis. 31, 159{168.
Kramer, H. P. & Bruckner, J. B. 1975 Iterations of a non-linear transformation for enhancement
of digital images. Pattern Recogn. 7, 53{58.
Lindebaum, M., Fischer, M. & Bruckstein, A. 1994 On Gabor s contribution to image enhance-
ment. Pattern Recogn. 27, 1{8.
Lindeberg, T. 1990 Scale space for discrete signals. IEEE Trans. Pattern Analysis Mach. Intell.
12, 234{254.
Marr, D. 1982 Vision. New York: Freeman.
Osher, S. & Rudin, L. I. 1990 Feature-oriented image enhancement using shock lters. SIAM J.
Numer. Analysis 27, 919{940.
Pearson, K. 1895 Contributions to the mathematical theory of evolution. II. Phil. Trans. R. Soc.
Lond. 33, 212{217.
Pollak, I., Willsky, A. S. & Krim, H. 1997 Scale space analysis by stabilized inverse di usion
equations. Lecture Notes in Computer Science, vol. 1252, pp. 200{211. Springer.
Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. 1992 Numerical recipes in
C, 2nd edn. Cambridge University Press.
Richards, J. I. & Youn, H. K. 1990 Theory of distributions. Cambridge University Press.
ter Haar Romeny, B. M. (ed.) 1994 Geometry-driven di usion in computer vision. Dordrecht:
Kluwer.
ter Haar Romeny, B. M., Florack, L. M. J., Salden, A. H. & Viergever, M. A. 1994 Higher-order
di erential structure of images. Image Vis. Comput. 12, 317{325.
Torroba, P. L., Cap, N. L., Rabal, H. J. & Furlan, W. D. 1994 Fractional order mean in image
processing. Opt. Engng 33, 528{533.
Weickert, J. 1998 Anisotropic di usion in image processing. Stuttgart: Teubner.

Proc. R. Soc. Lond. A (2000)

Vous aimerez peut-être aussi