Noise Reduction and Object Enhancement in Passive Millimeter Wave Concealed
Weapon Detection
Seungsin Lee†, Raghuveer Rao† and Mohamed-Adel Slamani‡
†
Rochester Institute of Technology, Rochester, NY 14623.
‡
ITT Industries, Rome NY 13441.
ABSTRACT
Passive MM wave offers the advantage of penetration for
concealed weapon detection. Its ability to penetrate through
fog, smoke, clothing etc. makes it an attractive candidate to
look for weapons concealed underneath a person’s
clothing. The sensor technology has advanced to a point
where it is possible to generate real time video sequences.
However, noise and blur are still severe problems. The
work reported here investigates the problem of
simultaneous noise suppression and object enhancement in
passive millimeter wave video sequences. The basis for the
approach is provided by undecimated wavelet transforms in
the spatial dimension and motion compensated filtering in
the temporal dimension. The paper presents the underlying
principles of the approach as well as experimental results.
1. INTRODUCTION
Passive millimeter wave (MMW) sensors measure the
apparent temperature through emitted or reflected energy
from sources. The output of the sensors is a function of the
emissivity of the objects in the MMW spectrum of the
receiver. The MMW sensors have advantages such as all
weather operation, ability to penetrate fog, smoke, clothing
or foliage, and day or night operation [2]. These abilities
make MMW sensors attractive in military application such
as detecting targets and guiding missiles as well as civilian
application such as weather forecasting. Detection of
concealed weapons is one of the appealing applications.
Generally, metal objects such as guns have low emissivity
and high reflectivity at MMW frequencies. With this
property and its ability to penetrate clothing, it offers the
potential to detect concealed weapons underneath a
person's clothing. Moreover, advanced sensor technology
makes it possible to generate real time video sequences
from the MMW sensor outputs. However, noise and low
contrast are severe problems in the video sequences.
In this paper, we address the problem of the
simultaneous noise suppression and object enhancement in
passive MMW video sequences. The basis for the approach
is provided by motion compensation in the temporal
0-7803-7622-6/02/$17.00 ©2002 IEEE I - 509
dimension and multiscale edge representation obtained by
undecimated wavelet transforms in the spatial dimension.
This paper is organized as follows. Temporal denoising is
described in Section 2. Section 3 briefly reviews multiscale
edge representation of images. Wavelet transform based
denoising and object enhancement are described in Section
4. Experimental results are presented in Section 5. Finally,
concluding remarks are given in Section 6.
2. MOTION-COMPENSATED FILTERING
Denoising of the video sequences can be achieved spatially
or temporally. Whereas spatial denoising filters such as
Wiener filter or even median filters cause blurring of image
details, temporal approaches can reduce the blurring effect
on edges [9]. In these methods, noise reduction is
performed along several consecutive frames. The simplest
temporal filtering is frame averaging, where the pixels at
the same coordinates in the frames are averaged without
further processing. This method is efficient in static parts of
the frames but may produce smearing in the moving areas
or spatial blurring when there are sudden temporal
variations. Motion-compensated filtering makes use of the
motion trajectory of a pixel to reduce the blurring effect
caused by the motion of an object between the frames. The
methods assume that the variation of the pixel gray levels
over any motion trajectory is due to noise (Figure 1). Thus,
applying a 1-D denoising filter to pixels along the
estimated motion trajectory can reduce the noise. In
addition to the temporal methods, adaptive filtering can be
applied for more robust denoising.
In this paper, we exploited motion-compensated
filtering for temporal noise filtering. To do this we choose
5 frames from N - 2 to N + 2. where Nth frame is the frame
to be denoised. Because of the low resolution, we
compared pixels of the frames to estimate the motion
trajectory instead of using blocks. Also we only implement
the comparison within 8 neighbors of the next or previous
frame because the movement of the sequence is usually not
much abrupt. Motion trajectory has been estimated by
comparing a pixel with the pixels in eight neighbors of the
pixel at the same coordinate in the previous frame and the
next frame. The errors between those pixels are computed,
IEEE ICIP 2002
and the pixel with minimum error is chosen. Hence, we 1 § x y ·
ψ 2k ( x, y ) = ψ¨ , ¸ , k = 1, 2 . (2)
have 5 pixels to be averaged along the estimated motion j
2j © 2j 2j ¹
trajectory. Chosen pixels from 5 frames are assumed to The dyadic wavelet transform of a function f(x, y) at (x, y)
represent the same pixels when there is no noise. These are is defined as
then averaged. W2k f ( x, y ) = f *ψ 2k ( x, y ) , k = 1, 2
j (3) j

x1 Mallat and Hwang [6] proved that a vector at point (x, y)

x2 composed by the dyadic wavelet transform pairs represents
the gradient of f ( x, y ) smoothed by θ ( x, y ) . That is,
x3
∇ 2 j f ( x, y )  ª¬W21j f ( x, y ) ,W22j f ( x, y ) º¼
1
∇ (θ 2 j ∗ f ) ( x, y )
Frame 1 Frame 2 Frame 2
pixel: x1 pixel: x2 = x1+ n pixel: x3 = x2+ n =
(4)
22 j
1
= 2 j ∇f ∗θ 2 j ( x, y )
Figure 1. Motion-compensated filtering along the motion 2
trajectory. Where n is noise. Denoised pixel value is The original function f ( x, y ) can be reconstructed from
approximated by averaging 3 pixels along the trajectory. W2kj f ( x, y ) with the inverse wavelets χ 2k j whose Fourier
transform satisfies
3. MULTISCALE EDGE REPRESENTATION VIA ∞

WAVELET TRANSFORM ¦ ª¬Ψ ( 2 ω

1 j
x , 2 j ω y ) Χ1 ( 2 j ω x , 2 j ω y )
j =−∞ (5)
Combination of spatial approach with temporal methods + Ψ ( 2 ω x , 2 ω y ) Χ ( 2 ω x , 2 ω y ) º¼ = 1
2 j j 2 j j

can reduce noise further. For the spatial method we The reconstruction formula is
employed the wavelet transform denoising method ∞

proposed by Bronnikov and Duifhuis [1]. The method is f ( x, y ) = ¦ (W f ( x, y ) ∗ χ

j =−∞
1
2j
1
2j
+ W22j f ( x, y ) ∗ χ 22j ) (6)
based on multiscale edge representation of images using
Detailed algorithms for the wavelet transform and
wavelet transform in the Mallat and Hwang's framework
inverse transforms are found in many other documents such
[5]. Unlike spatial filters such as an averaging filter or
as [6-8]. We utilize the Mallat's algorithm to compute
Weiner filter, the wavelet transform method provides more
undecimated fast wavelet transforms with the quadratic
flexibility and selectivity with less blurring effect.
spline wavelet whose Fourier transform is defined as
Furthermore, the wavelet transform approach provides a
sin 3 (ω x / 2 ) sin (ω y / 2 )
3
way to enhance objects in low contrast images. The method Ψ1 ( 2ω x , 2ω y ) = 4i exp ( −iω x ) (7)
(ω x / 2 ) (ω y / 2 )
3 3
has been used effectively in enhancing medical images [4],
x-ray imaging and tomography [1], and astronomical Impulse responses of the corresponding filters are found in
images [3] to denoise and enhance contrast in the images. [6, 7].
Object enhancement of the individual frame of the Multiscale edge information of the image can be
low contrast video sequence can also be performed from obtained by the magnitude and angle of the gradient
the multiscale edge representation. Such representation can
be obtained through undecimated wavelet transform j {
ρ 2 f ( x, y )  ª¬W21 f ( x, y )º¼ + ª¬W22 f ( x, y ) º¼
j
2
j }
2 1/ 2
(8)
proposed by Mallet and Hwang [5, 6]. They showed that
ªW22j f ( x, y ) º
the singularities in 1 or 2-D signals can be obtained by the θ 2 f ( x, y )  arctan « ». (9)
¬«W2 j f ( x, y ) ¼»
j

dyadic wavelet transforms. Let a function θ ( x, y ) be a

1

smoothing function. The smoothing function has the For each scale 2j, a set of local maxima of gradient points
integral one over x and y, and converges to zero when x and (xi, yi) collected by
y is infinity. The function is also supposed to be
differentiable. Then, the corresponding wavelets ψ 1 ( x, y ) G2 j ( f ) =
and ψ 2 ( x, y ) are defined by °­ ρ 2 j f ( xi , yi ) has local maximum at °½
® ª¬( xi , yi ) , ∇ 2 j f ( xi , yi ) ¼º | ¾
1 ∂
ψ ( x , y ) = φ ( x, y ) , ¯° ( xi , yi ) along the dicrection θ 2 j f ( xi , yi ) ¿
°
∂x (10)
(1)
∂
ψ 2 ( x, y ) = φ ( x, y ) represent the edge information at a scale j. For a J-level 2-
∂y D wavelet transformation, the collection
Let ψ 2k j ( x, y ) be the dilation of ψ ( x ) by a factor 2j
{S 2j
f ( x, y ) , ª¬G2 j ( f ) º¼
1≤ j ≤ J } (11)

I - 510
where S2 f ( xi , yi ) is the discrete low-pass approximation
J Enhancement of the contrast of an image is now
of f ( x, y ) at the coarsest scale 2 , is called the multiscale
J performed with the denoised edges G 2 ( f ) by transforming
j

edge representation of the image f ( x, y ) . the magnitude ρ 2 f ( x, y ) at the edge point (x, y). There are
j

Mallat and Zhong [6], and Mallat and Hwang [5]
showed that the original image can be estimated from the
multiscale edge representation by the iterated projection
onto the convex sets (POCS) methods. However, we use all
the available details as well as the multiscale edge
representations to reconstruct denoised and enhanced
images. This modified approach removes the need of
iterated projection on the convex sets and increases the
computational speed.
4. DENOISING AND OBJECT ENHACEMENT
USING THE MULTISCALE EDGE
REPRESENTATION
With wavelet transformation, higher frequency components
are generally located in finer scales, the edges in each scale
are represented the set G2 ( f ) in (10). Therefore, most of
j
the noise will be observed at edge points in the fine scales
because noise generally exhibits high-frequency
characteristics. By suppressing the noisy edges in the fine
scales, noise can be reduced without removing true edges.
Denoising with wavelet transforms can be performed by
two ways, either using the magnitude (modulus based
method) or the angle information (phase based method) of
the gradient on the multiscale edge representation [1]. The
modulus-based methods use information about the
magnitude of the gradient. It is supposed that noise
typically has a small gradient than true edges. Thus, noise
can be removed by eliminating the gradients whose
magnitudes are smaller than a given threshold. For the
phase based algorithms, an observation that the gradients of
true edges have similar angles in a small neighborhood is
used. A gradient is considered as noise if its angle is totally
different from the neighbors. We adopted the modulus
based method. A problem with the modulus based methods
is that if the same threshold is applied to all the scales, it
may remove true edges as well as noisy edges in lower
scales because the differences between the magnitude of
noisy and true edges become smaller in lower scales. To
avoid this problem, we apply a variable threshold
t j = k × max ª¬ ρ 2 f ( x, y ) º¼ , 0 < k < 1
j (12)
Applying this method, we assume that only
G 2 j ( f )
(13)
{ }
= ª¬( xi , yi ) , ∇ 2 j f ( xi , yi ) ¼º | ρ 2 j f ( xi , yi ) > t j , ∀i ∈ I 2 j ( f )
have the true edges and other values are considered as
noise and removed.
two transforming methods for the enhancement, which are
the linear and non-linear transformations [1, 3]. The linear
transform is performed by multiplying the multiscale
gradients by a constant factor. The method can be modified
to use different multiplication factors for different scales.
On the contrary, multiplication factors of non-linear
transfer function methods are variable with respect to the
magnitude of the gradient. In this paper we used the non-
linear transform
°­ ª g º ½°
G 2 j ( f ) = ® « k − ( k − 1) » × g | ∀g ∈ G2 j ( f ) ¾ (14)
max g
¯° ¬ ¼ ¿°
where, k is a constant and g is a gradient at an edge.
Equation (14) reveals that the multiplication factor
becomes small, when the gradient is large.
Transformed edges by (13) and (14) are used to
reconstruct the denoised and enhanced image using (6). As
mentioned above, instead of using iterated method with
only the edge information, other gradient information is
used to reconstruct the enhanced image.
5. EXPERIMENTAL RESULTS
In this section, we present examples of the denoised and
enhanced frames from the video sequences captured by the
passive millimeter wave array sensor. The sensor used here
is a passive millimeter-wave camera manufactured by Trex
Enterprises. The structure of the camera is based on a
phased radiometer array. The output of the camera is
converted into a 60 x 75 pixel digital image frame . Output
video sequence has been preprocessed by the following
procedures;
1) Gain and offset calibration.
2) Linear interpolation from adjacent good pixels to correct
isolated dead pixels.
3) A previously saved background image was then
subtracted from each frame in real time to remove the
effects of background objects such as buildings, cars etc.
4) Pixel values of the original data represent a value in a
scale of -90 to –70 in this example. The values were
linearly mapped into [0, 1].
Figure 2 shows two frames taken from the sample
video sequence. In the frames, the concealed gun does not
show clearly because of noise and low contrast. To reduce
the noise in the scene, we first applied motion–
compensated filtering. The images in Figure 3 show
denoised frames by the motion compensated filtering. The
frames were then spatially denoised and enhanced by the
wavelet transform methods. The frames were decomposed
with 4 decomposition level and edges in the fine scales
were detected using the magnitude and angles of the

I - 511
gradient of the multiscale edge representation. The
threshold for denoising was 15% of the maximum gradient
at each scale. Figure 4 shows the final results, which show
the contrast enhanced and denoised frames. One can notice
the image of a pistol in the chest of the subject is more
apparent in the enhanced frames than in the original
frames.
Figure 2. Original frames
Figure 3. Denoised frames by averaging along the motion
trajectory
Figure 4. Denoised and enhanced frames by wavelet approach
6. CONCLUSION
With the abilities of penetration and high reflectance off
metal surfaces, MMW sensors become attractive tools for
detection of concealed weapons. Moreover, sensor
technologies now make it possible to generate the video
sequence from the sensor output in real time. However, the
output video sequences have problems related to noise and
blurriness. We have adopted undecimated wavelet
transform and motion-compensated filtering to solve to the
problems. It was possible to reduce the noise in the video
sequences without much blurring of edges. Additionally,
objects in the low-contrast frames could be enhanced using
the multiscale edge representation obtained by the wavelet
transform. The paper also provided experimental results of
the algorithms. For further research, there is one more
noticeable problem with the video sequence, which is that
ghost images due to grating lobes and side lobes exist in the
frames. The ghost images are generated because of the
phased array structure of the MMW sensor. The algorithm
in the paper enhanced the unwanted ghost images as well
I - 512
as true objects. Techniques are currently being investigated
to address this problem.
ACKNOWLEDGEMENT
This work is sponsored in part by the National Institute of
Justice under an Air Force Research Laboratory, Rome,
contract to support detection of concealed weapons.
Additional support was provided by NYSTAR. Trex
Enterprises supplied the passive millimeter wave video
sequences.
REFERENCES
[1] A. V. Bronnikov and G. Duifhuis, "Wavelet-Based
Image Enhancement in X-Ray Imaging and
Tomography," Applied Optics, vol. 37, no. 20, pp.
4437-4448, 1998.
[2] L. A. Klein, Millimeter-Wave and Infrared Multisensor
Design and Signal Processing. Boston: Artech House,
1997.
[3] J. Lu and J. Dennis M. Healy, "Contrast Enhancement
Via Multiscale Gradient Transformation," pp. 482-486,
1994.
[4] J. Lu, J. Dennis M. Healy, and J. B. Weaver, "Contrast
Enhancement of Medical Images Using Multiscale Edge
Representation," Optical Engineering, vol. 33, no. 7, pp.
2151-2161, 1994.
[5] S. Mallat and W. L. Hwang, "Singularity Detection and
Processing with Wavelets," IEEE Transactions on
Information Theory, vol. 38, no. 2, pp. 617-643, 1992.
[6] S. Mallat and S. Zhong, "Characterization of Signals
from Multiscale Edges," IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 14, no. 7, pp.
710-732, 1992.
[7] R. M. Rao and A. S. Bopardikar, Wavelet Transforms :
Introduction to Theory and Applications. Reading, MA:
Addison-Wesley, 1998.
[8] G. Strang and T. Nguyen, Wavelets and Filter Banks.
Wellesley, MA: Wellesley-Cambridge Press, 1997.
[9] A. M. Tekalp, Digital Video Processing. Upper Saddle
River, NJ: Prentice-Hall, Inc., 1995.