Fuzzy Adaptive Resonance Theory Neural Networks

Fuzzy Adaptive Resonance Theory Neural

Networks
Dr. Carey Schwartz

Technical Objective/Expected Payoff

Technical Background and Proposed Work
Summary of Technical Accomplishments Since Last Proposal
Summary of Qualifications/Experience of Key Personnel

Technical Objectives/Expected Payoff:

The primary objective of this task is to perform simulations which evaluate the effectiveness of a recently-
developed class of neural network algorithms, based on fuzzy logic and adaptive resonance theory (ART),
for difficult application areas in which traditional methodologies have proven inadequate. Chief among
these applications are automatic control, and identification/classification of signals in noisy environments
which is required for identification of friend, foe, neutral, target classification, and affordable, rapid
development of robust control systems, as well as Battle Damage Indication and Automatic Target
Cueing. The latter includes: identification of radar transmitters from radar pulse shape; detection of the
presence of signals in noise, e.g., radar reflections of sea-skimming missiles, and non cooperative
automatic target recognition (ATR), particularly of aircraft, using both ordinary and enhanced downrange
radar profiles. We will also explore improvements to the underlying algorithms of relevance to these and
a wide range of other applications. These improvements include confidence measures, by means of which
the system can indicate the degree of certainty with which it is identifying any given input. Valid
confidence measures can be employed by neural networks which must take risk into account when
performing control tasks. In target or signal identification applications, confidence measures can be of use
in preventing "friendly-fire" kills, as well as emitter identification. Also of use for this purpose are
unknown-target rejection mechanisms, by means of which the system avoids unjustified and inaccurate
"guessing" when attempting to identify a previously-unencountered signal or target. We will also explore
noise-reduction algorithms specifically tailored to fuzzy ART-based systems, and will continue
discussions with others regarding the implementation of fuzzy ART-based neural network architectures in
VLSI hardware.

Technical Background and Proposed Work:

Adaptive Resonance Theory (ART) 1, the fundamental unsupervised-learning module which lies at the
core of ART neural networks, employs recurrent connections in order to implement a particularly efficient
type of clustering algorithm, one which in many ways imitates the operation of biological memory.
Events which are sufficiently similar to previously-learned categories of events are grouped with the
category which they most resemble; stored data is modified appropriately, but the amount of data which is

http://www.nawcwpns.navy.mil/hybrid/fartnn.html (1 of 5) [12/11/2001 12:09:37 AM]

 Fuzzy Adaptive Resonance Theory Neural Networks

stored does not increase. Novel events, however, generate new categories and are remembered without
degrading the recollection of other categories already in memory at a fixed additional cost of memory.

In ART 1 the degree of novelty required to trigger the shift between these two modes of remembering is
governed by a vigilance parameter, the value of which is set externally. ARTMAP incorporates ART 1
into a supervised-learning system, and turns the vigilance parameter into a dynamical variable which
increases or decreases in response to the degree of difficulty presented by the pattern-recognition task in
different parts of configuration space. This leads to the effective partitioning of the input space with a
variable-size grid and tends to minimize data storage requirements. The use of fuzzy logic at the level of
single-neuron processing and weight adaptation provides a mechanism by which the fuzzy ART and fuzzy
ARTMAP algorithms can operate on real-valued inputs without undue increase in complexity. Both ART
1 and ARTMAP operate on binary data. A recent extension of the fuzzy ARTMAP algorithm, ART-
EMAP, allows classification decisions to be based on sequences of observations. In the same way that,
e.g., a human will ask for repetition of a word heard over a noisy telephone line, ART-EMAP can delay
identification of its input until additional information is gathered. This can lead to a substantial increase in
accuracy over single-observation classification.

Control problems can be viewed as functional mappings; i.e., a neural network learning to control an
external system must learn a multidimensional function whose independent variables represent the state of
the system to be controlled, including possibly its past history, and whose dependent variables represent
the control action to be taken. To address these issues, we have identified a number of sub tasks which are
identified below.

Control problems. Fuzzy ARTMAP is capable of learning to approximate continuous functions, and
recently-published developments indicate how to improve the smoothness of the approximation. In light
of the success of ART-based networks at pattern-recognition tasks, including but not limited to the
present ATR task, their application to control problems must be viewed as a promising and as-yet under
explored area of inquiry. A control problem of moderate complexity for which an existing solution exists
will be employed to evaluate the application of fuzzy ARTMAP for control. Comparisons will be made
with regard to computational time required to implement the controller and performance.
Recommendations with regard to required basic research as well as problems for which Fuzzy ARTMAP
represents an affordable solution technique will be identified. A down selection from this problem set wll
be made, and a controller developed for use in a system.

Confidence measures. It has been shown that fuzzy ARTMAP can be trained in such a manner as to yield,
during post-training operation, the relative probabilities that a pattern to be classified falls into each of the
possible classes. We will test the effectiveness of using the ratios of relative-probability estimates as
confidence measures. ART-EMAP performs classification based on sequences of observations, and each
observation in the sequence can yield a separate tentative classification. The distribution of these tentative
classifications may also used to yield a confidence measure; the potential of this approach will be
investigated, and its effectiveness compared with alternatives. Potential areas of application for this
technology include Battle Damage Indication and Automatic Target Cueing.

http://www.nawcwpns.navy.mil/hybrid/fartnn.html (2 of 5) [12/11/2001 12:09:37 AM]

 Fuzzy Adaptive Resonance Theory Neural Networks

Unknown-target rejection. As we showed in previous work, fuzzy ARTMAP can learn to distinguish
between targets with which it is familiar and those which it has not "seen" during training. The technique
which was employed, increasing the baseline vigilance parameter, caused a significant increase in the
storage requirements of the systems. We will investigate the use of one or several of the confidence
measures as cutoffs to separate familiar from unfamiliar targets in a more computationally-efficient
manner which minimizes storage requirements.

Noise immunity. A preprocessing technique know as complement coding is typically used in conjunction
with fuzzy ART, fuzzy ARTMAP and ART-EMAP to produce increased coding efficiency. A
modification, partial feature vector complement coding, can be employed to allow the network to make its
classification decisions based on a subset of the components of the full feature vector. One potential
application of partial feature vector complement coding is the removal of those components of the feature
vector which are comparable in magnitude to the ambient noise level, or whose magnitude is smaller than
some fraction of that of the largest component. We will investigate this approach as a means of improving
the classification accuracy of fuzzy ARTMAP and ART-EMAP in the presence of noise. In addition, a
feature of the ART-EMAP algorithm, distributed F2a activation, is likely to improve performance in
noisy environments, and will be tested in upcoming simulations.

Detection of signals in noise. Related to the problem of identifying the nature of a signal buried in noise is
that of determining whether a signal of interest is present in a noisy input stream in the first place. This
may be viewed as a two-category classification problem, the two categories being "signal-present" and
"signal-absent." Application of the increased-baseline-vigilance to this problem proved ineffective; we
will examine whether fuzzy ARTMAP networks in conjunction with suitable confidence measures and
noise reduction modifications can perform this task.

Radar pulse shape identification. Different radar transmitters produce pulses of different shapes. The
shape of the pulse may therefore be used to identify the type of radar transmitter which produced it. Tests
on identification of real radar pulse data by fuzzy ARTMAP will be performed. The use of confidence
measures and noise-reduction techniques will also be investigated in this context.

Other preprocessing techniques. Enhanced range profiles are range profiles constructed from time-
integrated direction-of-arrival information as well as amplitude data. Fuzzy ARTMAP and ART-EMAP
will be tested as classifiers on enhanced range profiles. ART-EMAP inherently makes use of time-
integrated information, and so may be applied as well to profiles which incorporate direction-of-arrival
information but have not been processed into enhanced form. This work will be performed in conjunction
with the Hybrid Signal Processing Radar Based Target Identification Proposal. Testing on range profiles
expressed in terms of wavelet-expansion coefficients will also be investigated.

Improvements in range profile and target simulation. Two improvements in the simulations of target-
identification training and testing scenarios will be implemented. Variations in elevation will be included,
and the XPATCH software, which generates validated radar range profiles, will be obtained and used in
evaluation of the neural network. In addition, critical experiments employing operating radars will be
designed.

http://www.nawcwpns.navy.mil/hybrid/fartnn.html (3 of 5) [12/11/2001 12:09:37 AM]

 Fuzzy Adaptive Resonance Theory Neural Networks

Summary of Technical Accomplishments Since Last Proposal:

In collaboration with Stephen Grossberg, we have developed and investigated a technique, reset buffering,
for achieving even more efficient data storage. Whereas the basic fuzzy ARTMAP architecture resets data-
storage nodes whenever a template mismatch occurs, buffered fuzzy ARTMAP uses information
regarding the statistics of the training process to reduce the frequency of resets, thereby minimizing the
recruitment of new nodes and the total amount of stored data. Considerable effort has been expended in
bringing the electromagnetic simulator xpatch on line. A number of these difficulties arose from
inadequate documentation and have been documented and reported to the authors of the code. The
buffering technique has been tested on simulated range profile datasets of the type described in [1]. In
addition, it has been tested on smaller datasets generated using the "xpatch. This work will be reported in
[2]. These results indicate that the scheme proposed for reduction of library size should be implementable
and will succeed.

Schedule slippage has occurred due to two phenomena. The first is to Dr. Rubin assuming a temporary
position at Boston University within the group of Dr. Stephen Grossberg and Dr. Gail Carpenter, the
developers of the ART family of networks. This resulted in restructuring of the task and required the
award of a contract to Boston University to support Dr. Rubin. The contract was finally awarded in March
1996. As a result of restructuring a post-doctoral fellow is being recruited to be employed at Naval Air
Warfare Center Weapons Division. The nature of the postdoctoral program only allows offering positions
4 times a year on a quarterly basis.

Publications, Presentations, and Patents:

1. Rubin, M.A. "Application of Fuzzy ARTMAP and ART-EMAP to Automatic Target Recognition
Using Radar Range Profiles,"Neural Networks, vol 8, pp1109-1116 (1995).
2. Grossberg, S., Rubin, M.A., and Streilein, W.S. "Buffering and Memory Efficiency in a Fuzzy
ARTMAP Target Recognition Application," in preparation. To be submitted to ANNIE '96.

Performer:
Dr. Carey Schwartz
Code 474400D
NAWCWPNS
China Lake, CA 93555-6001
Phone: (760)939-1442
FAX: (760)939-1409

http://www.nawcwpns.navy.mil/hybrid/fartnn.html (4 of 5) [12/11/2001 12:09:37 AM]

 Fuzzy Adaptive Resonance Theory Neural Networks

schwartzc@navair.navy.mil

Summary of Qualifications/Experience of Key Personnel:

Dr. Carey Schwartz obtained his Ph.D. In physics from the Pennsylvania State University. Upon
completion of the doctorate he held postdoctoral fellowships in physics at Indiana University and in
chemistry at the University of Waterloo (Waterloo, Ontario, Canada). He then assumed a staff position in
the Semiconductor and Surface Physics Branch of the Physics Division of the then Naval Weapons Center
(now Naval Air Warfare Center Weapons Division). Dr. Schwartz subsequently became the head of the
Sensors and Signals Branch of the Physics Division, and has since become the head of the Computational
Sciences Branch of the Research and Technology Division at NAWCWD. Since 1991 he has been the
Laboratory Project Manager for what was then the Hybrid Signal Processing Block and is currently the
Laboratory Project Manager for the Hybrid Signal Processing Technology Plan which includes work in
neural networks, wavelets, signal processing hardware, and radar based target identification. Dr. Schwartz
is the author (co-author) of over 35 papers in a variety of fields, holds one issued patent, has two patent
applications pending, and has made over 50 conference/seminar presentations. Dr. Schwartz has also
spent a 6 month tour at the Office of Naval Research in the Strike Technology Division (Code 35) where
he formulated and as a collateral duty will manage a new basic research program which will begin
execution in FY97.

Dr. Mark A. Rubin obtained his Ph.D. In physics from the University of Chicago. He has done
postdoctoral research at the University of Texas at Austin in the group of Nobel Laureate Steven
Weinberg and at Fermi National Accelerator Laboratory. He has been an assistant professor of physics at
Rockefeller University, and vice-chair of the Section of Physics and Astronomy of the New York
Academy of Sciences. He originated the notion of applying fuzzy ARTMAP to the range profile ATR
problem, developed all the code, and carried out all the investigations on this subject to date. He has
presented refereed papers on neural network software development and ATR applications at three
international conferences (ICANN in 1993, Amsterdam; WCNN in 1994, San Diego; ICNN in 1994
Orlando). His prior research involved analytical work and extensive code development for finite-element
computation in quantum mechanics, as well as investigations of the superstring and Kaluza-Klein unified
field theories.

Return to Hybrid Signal Processing Technology

Wed Nov 27 14:13:40 PST 1996

http://www.nawcwpns.navy.mil/hybrid/fartnn.html (5 of 5) [12/11/2001 12:09:37 AM]