Data fusion

Piero Zappi pzappi@deis.unibo.it DEIS Universit di Bologna

Outline
WSN overview
Motivation Application Design constraints

Multisensor data fusion

Introduction and definition Advantages Applications

Techniques and examples

Direct fusion of sensors data Features analysis High level decisions

WSN overview
A solution designed to collect information from the environment Flexible
Environment may change Nodes may fall Data need may grow

Efficient
Often must be real time Reliable Deeply integrated within the environment

Cheap

WSN overview
Security
Surveillance Object/Human tracking

Military
Vehicle tracking Anti intrusion

Environmental
Precision agriculture Biological environment monitoring

Biomedical
Patients care Biofeedback

Domotic
Home automation

WSN overview
Low power
Last months or years with normal batteries Energy scavenging

Low cost
Cheap components Self organizing Easy to deploy and maintain

Low data throughput

Analyze data on local node instead of sending raw samples

Latency
Some application require real-time data report

Reliability
Fault tolerant (redundancy)

Security

Multisensor data fusion

Something new?
As human and animals evolved they always used data fusion to survive
Vision and sense of hearing Smell and touch

Now the technological progress in

Processing techniques New sensors Hardware

have provided the capability to emulate, in hardware and software, the natural data fusion capabilities of human and animals

Multisensor data fusion

Definition : Data fusion is the process of combining data or information to estimate or predict entity states Combine data
From sensor of the same kind Form different sensors

Estimate or predict the state of some aspect of the universe

Multisensor data fusion

Data fusion techniques combine data from multiple sensors to achieve more specific inferences than could be achieved by using a single, independent sensor Multiple sensors reduce uncertainty Redundant information increase reliability Complementary information allow to perceive features not detectable by single sensors (improve observability)

Multisensor data fusion

Data fusion process shows several difficulties Representing the uncertainty in observations and model
Non deterministic approach

Combining noncommensurate information

Signal from different sensor Discrete and continuous attributes

Handle data
Large number of observations Different way of associating and interpreting them

Multisensor data fusion

Military applications: Location and characterization of enemy units and weapons High level inferences about enemy situation
Relationship among units Air to air or surface to air defense Ocean monitoring Battlefield intelligence Strategic warning
Nuclear threats

Multisensor data fusion

Non military applications: Condition based maintenance
Detection of system faults

Robotics
Object localization Object recognition Guide to locomotion

Medical
Location and identification of diseases

Environmental monitoring
Location and identification of natural phenomena

MDF Techniques
Two fundamental processes: Sensor measuring
Obtain measurements of quantities Depend on the observed phenomenon

Feature extraction
Derive item of information from sensor data Highly influence the correct estimation of event

MDF Techniques
Three basic alternatives Direct fusion of sensor data

MDF Techniques
Three basic alternatives Direct fusion of sensor data Representation of sensor data via features vector and fusion of features

MDF Techniques
Three basic alternatives Direct fusion of sensor data Representation of sensor data via features vector and fusion of features Integration of high-level inferences from each sensor

MDF Techniques
Direct fusion of sensor data

fusion of sensor data

Used when multisensor data are commensurate

Measure the same physical phenomena

Sensor data are directly combined

Signals modeled as random variables Uncorrelated noise corrupt data

Require high synchronization

Temporal Spatial

Use of Bayesian filtering

MDF Techniques
Gaussian approximation
Kalman Filter Extended Kalman Filter Unscented Kalman Filter Grid Based methods

fusion of sensor data

Sequential Monte Carlo methods

Particle Filter Unscented Particle Filter Mixed

Complementary filters
Complementary Kalman Filter

MDF Techniques

fusion of sensor data

Kalman filtering address the general problem of trying to estimate the state of a discrete time controlled process. Estimate a process using a recursive algorithm
Estimate the process state at a certain time (predict) Obtain feedback from noisy measurement (correct)

MDF Techniques

fusion of sensor data

Equations falls into two groups: Time update

Estimate future state Obtain an a priori estimation

Measurement update
Are responsible of the feedback Obtain an a posteriori estimation
Time update (Predict) Measurement update (Correct)

MDF Techniques
Combines all available data
Regardless of their precision Regardless of noise

fusion of sensor data

Uses prior knowledge of the system and measurement devices

Dynamics Inaccuracies

Uses statistical description of noise errors and uncertainties Uses information of initial condition

10

MDF Techniques

fusion of sensor data

Example: human motion capture

Humans are seen as composition of rotational joint Conventional method uses gyroscope or accelerometers to detect position of each segment
Both estimate orientation of each segment Use of only one kind of sensor results in low performance

Kalman filters are used to combine both sensors

Increase reliability Decrease uncertainty

MDF Techniques representation via features vectors

Representation of sensor data via features vector Involve the extraction of representative features from sensor data
Mean value Variance Number of occurrences

Features are matched to corresponding features in a symbolic world model (pattern recognition) Humans use feature based cognitive function

11

MDF Techniques representation via features vectors

Pattern recognition: the act of taking in raw data and taking an action based on the category of the data
Segmentation

Isolate single patterns

Feature Extraction

Extract most representative features

Classification

Determine input pattern category

MDF Techniques representation via features vectors

Design Cycle
Collect Data

Collect an adequately large and representative amount of data Choose features to extract from raw data (strongly problem dependent) Choose models to describe classes, typically in mathematical form Use collected data to determine the classifier Measure the error rate and switch from one set of features to another one

Choose Features

Choose Model

Training

Evaluation

12

MDF Techniques representation via features vectors

Features choice
Strongly impact on system performance Must characterize classes Any rule of thumb

Training (Learning)
Fundamental to produce a good classifier Must provide good generalization

MDF Techniques representation via features vectors

Metric methods
Non parametric techniques
Parzen window Kn nearest neighbor estimation The nearest neighbor estimation

Parametric techniques
Linear discriminant functions and decision surfaces Support vector Machines (SVM)

Multilinear neural networks

Non metric method

Decision Trees Grammatical methods

13

MDF Techniques representation via features vectors

Example: human tracking system
Recognize how an individual is walking
Stop Standing Slow/Fast forward

Wireless sensor nodes placed over the ankle mounting an accelerometer 4 representative features
Mean Variance Energy Correlation

91% average correct detection

MDF Techniques integration of high level inferences

Integration of high-level inferences Each sensor made a preliminary determination of some attributes
Location Identity Behaviors

Successive integration of high abstraction level information Use of pattern recognition, Bayesian estimation and fuzzy logics

14

MDF Techniques integration of high level inferences

Pattern Recognition
High level information are seen as features All previous method can be used

Bayesian estimation
Combines information according to rules of probability theory

Fuzzy logic
Analyze information with If-Then logic statement Weight contribution according to their degree of truth Combined weighted contributions

MDF Techniques integration of high level inferences

Example: road boundary detection
Driver assistance
Alarm in case of car driving off the road

Threat detection
Alarm in case of object within the driving lanes

Fusion of radar and vision system placed on the vehicles Several sources of noise
Shadows Poor radar SNR

15

Conclusion
Technological progress provided the capability to emulate the natural data fusion capabilities of human and animals of fusing data from different sensor Data fusion can be performed at three levels
Sensor data Features vectors High level inferences

Several techniques has been developed to process data fusion at each level

16