Académique Documents
Professionnel Documents
Culture Documents
unmixing
Outlines
gipsa-lab
Spectral mixture
Some particularities of hyperspectral data are not to be found in other types of image data:
Mixed pixels
Sub-pixel targets
gipsa-lab
Spectral mixture
4000
Reflectance
3000
0
300 600 900 1200 1500 1800 2100 2400
Wavelength (nm)
4000
Reflectance
Pure pixel
3000
(water) 2000
1000
0
300 600 900 1200 1500 1800 2100 2400
Wavelength (nm)
5000
Reflectance
4000
Mixed pixel
3000
(vegetation + soil) 2000
1000
0
300 600 900 1200 1500 1800 2100 2400
Wavelength (nm)
Mixed pixel:
The signal detected by a sensor into a single pixel is frequently a combination of
numerous disparate signals.
gipsa-lab
Spectral mixture
Usually one of the reason why mixed pixels exist depends on one of two reasons:
gipsa-lab
Spectral mixture
In a hyperspectral image, each pixel, depending on the spatial resolution of the sensor, cover a certain area of
the ground. The detected signal is a combination of the signals produced by the different types covered by the
pixel
1
Detected
signature
gipsa-lab
Linear spectral mixture
If the endmembers in a pixel appear in spatially segregated patterns similar to a square checkerboard, these
systematics are basically linear. In this case the spectrum of a mixed pixel is a linear combination of the
endmember spectra weighted by the fractional area coverage of each endmember in a pixel.
M
x = ai si + w
i=1
Linear mixture
gipsa-lab
Linear spectral mixture
E0 E 0e 1 .... E e N
0
E S = E 0 e k
k =1
If the total surface area is considered to be
proportionally between the different
divided
materials in the within the pixel then the
reflected radiation will convey the
characteristics of the associated media with
the same proportions.
Target pixel
gipsa-lab
Linear spectral mixture
Mixed pixel
Linear mixture Vegetation (60%)
0.6*veg+0.4*build
Buildings (40%)
There exists a linear relationship
between the fractional
abundance of the substances
comprising the area being
imaged and the spectra in the
reflected radiation.
gipsa-lab
Non linear spectral unmixing
If the components of interest in a pixel are in an intimate association, like sand grains of different
composition in a beach deposit, light typically interacts with more than one component as it is multiply
scattered, and the mixing systematics between these different components are nonlinear.
Nonlinear mixture
gipsa-lab
Nonlinear spectral mixture
E0
N
k
E S = E 0e k=1
E 0e 1 E 0e N
Target pixel
gipsa-lab
Non linear spectral unmixing
Nonlinear mixture
Calcite Kaolinite
gipsa-lab
Nonlinear spectral mixture
gipsa-lab
1.9
Linear vs nonlinear mixing model
Linear mixture model
Assumes that endmember substances are sitting side-by-side within the FOV.
Nonlinear mixture model
Assumes that endmember components are randomly distributed throughout the FOV.
Multiple scattering effects.
gipsa-lab
spectral unmixing
The basic premise of mixture modeling is that within a given scene, the surface is dominated by
a small number of distinct materials that have relatively constant spectral properties.
These distinct substances (e.g., water, grass, mineral types) are called endmembers, and the
fractions in which they appear in a mixed pixel are called fractional abundances.
The goal is to find the endmembers that can be used to unmix other mixed pixels
gipsa-lab
spectral unmixing scheme
gipsa-lab
Dimensionality reduction
gipsa-lab
Dimensionality reduction
A dataset composed of hundreds of high correlated narrowband channels may cause problems in the:
gipsa-lab
Dimensionality reduction
gipsa-lab
Dimensionality reduction Hughes effect
Hughes demonstrated that having a fixed number of training samples, the ability of
a classification algorithm to correctly predict the effective result, decreases as the
input dimension increase.
Intrinsic dimension
Any low-dimensional data space can trivially be turned into a higher dimensional
space by adding redundant or randomized dimensions, from this assumption many
high-dimensional data sets can be reduced to lower dimensional data without
significant information loss.
Hughes, G.F., 1968. "On the mean accuracy of statistical pattern recognizers", IEEE Transactions on Information Theory,
IT-14:55-63.
gipsa-lab
Feature selection/extraction
Feature
reduc<on,
or
dimensionality
reduc<on,
is
the
process
of
reducing
the
number
of
features
of
the
considered
data.
feature selec/on algorithms: Selec<on of a sub-op<mal subset of the original set of features
feature
extrac/on
algorithms:
Projec<on
of
the
original
feature
space
into
a
lower
dimensional
subspace
that
preserves
most
of
the
informa<on
gipsa-lab
Feature selection
Feature selection algorithms perform a search through the space of feature subsets, and, as a
consequence, must address four basic issues affecting the nature of the search:
1. Starting point. Selecting a point in the feature subset space from which to begin the
search can affect the direction of the search. One option is to begin with no features and
successively add attributes. In this case, the search is said to proceed forward through the
search space. Conversely, the search can begin with all features and successively remove
them. In this case, the search proceeds backward through the search space. Another
alternative is to begin somewhere in the middle and move outwards from this point.
2. Search organization. An exhaustive search of the feature subspace is prohibitive for all
but a small number of features. With N initial features there exist 2^N possible subsets.
3. Evaluation strategy. A criterion to estimate the final accuracy of the feature subsets.
4. Stopping criterion. A feature selector must decide when to stop searching through the
space of feature subsets. Depending on the evaluation strategy, a feature selector might
stop adding or removing features when none of the alternatives improves upon the merit of
a current feature subset. Alternatively, the algorithm might continue to revise the feature
subset as long as the merit does not degrade.
gipsa-lab
PCA
Geometrically speacking, PCA change the reference system to enhance the data structure
gipsa-lab
MNF
MNF
is
a
linear
transforma<on
that
consists
of
two
separate
PCA
rota<ons
and
a
noise
whitening
step:
Use
the
noise
covariance
matrix
to
decorrelate
and
rescale
the
noise
in
the
data
(noise
whitening).
This
results
in
transformed
data
in
which
the
noise
has
unit
variance
and
no
band-to-band
correla<ons.
The
MNF
transform,
like
the
PCA
transform,
is
an
eigenvector
procedure,
but
based
on
the
covariance
structure
of
the
noise
in
the
image
data
set.
Goal
of
the
MNF
transform
is
to
select
component
in
a
way
that
maximizes
the
signal-to-
noise
ra<o
(rather
than
the
informa<on
content).
gipsa-lab
Independent
Component
analysis
Such a representation seems to capture the essential structure of the data in many
applications, including feature extraction and signal separation.
x j = a j1s1 + a j 2 s2 + ...+ a jn sn
gipsa-lab
Independent
Component
analysis
The
cocktail
party
problem
Imagine to be in a room where two people are speaking simultaneously. You have
two microphones, which you hold in different locations. The microphones give you
two recorded time signals, which we could denote by x1(t) and x2(t), with x1 and
x2 the amplitudes, and t the time index. Each of these recorded signals is a
weighted sum of the speech signals emitted by the two speakers, which we denote
by s1(t) and s2(t) . We could express this as a linear equation:
If we knew the parameters a, we could solve the linear equation by classical methods.
If you dont know the a, the problem is considerably more difficult.
gipsa-lab
Independent
Component
analysis
The
cocktail
party
problem
s1
s2
x1(t)
x2(t)
gipsa-lab
Independent
Component
analysis
We can use some information on the statistical properties of the signals s(t) to
estimate the a. To do this it is enough to assume that s1(t) and s2(t) , at each time
instant t, are statistically independent.
Original Reconstructed
Differences in
amplitude
ICA algorithms return signals that are very close to the original source signals
gipsa-lab
Nonlinear
Principal
Components
Analysis
gipsa-lab
Endmember extraction
gipsa-lab
Endmember extraction
Endmember extraction algorithms: simultaneous versus sequential
Different approaches depending on the actual implementation:
Simultaneous algorithms: they assume that the number of endmembers p is known
a priori. For each value of p, they recalculate all the endmembers (they do not take
advantage of previously calculated p-1 endmembers). Example: N-FINDR.
N-FINDR algorithm
gipsa-lab
Pixel Purity Index (PPI)
Parameters: k (number of skewers) and t (cut-off threshold)
It is not an iterative algorithm (no convergence criteria)
No estimation of number of endmembers a priori
Manual intervention required to select a final set of endmembers
gipsa-lab
Pixel Purity Index (PPI)
Pixel Purity Index (PPI) - Works by projecting each pixel onto one vector from a set
of random vectors spanning the reflectance space. A pixel receives a score when it
represent an extremum of all the projections. Pixels with the highest scores are
deemed to be spectrally pure.
Skewer
Extreme 1 Skewer
pixel 2
Extreme
pixel
Skewer
3
Extreme
pixel
Extreme
pixel
gipsa-lab
Pixel Purity Index (PPI)
gipsa-lab
Orthogonal Subspace Projection
Spectral-based method developed by Harsanyi and Chang (IEEE-TGRS 1994):
1. Select the brightest pixel from the scene as the initial one
2. Apply an orthogonal subspace projection operator to all the pixels in the original
image to identify the most orthogonal pixel to the first one (second endmember)
3. Apply the orthogonal subspace projector again to all pixels, looking for the most
orthogonal one with regards to the first two (third endmember)
.
4. Repeat the process until a predetermined number of endmembers is found
gipsa-lab
N-FINDR
Spectral-based method developed by Winter (Proceedings of SPIE, 2003):
1. Dimensional reduction using MNF (ordering resulting components in terms of
signal to noise ratio) or PCA (ordering components in terms of variance)
gipsa-lab
N-FINDR
Spectral-based method developed by Winter (Proceedings of SPIE, 2003):
1. Dimensional reduction using MNF (ordering resulting components in terms of
signal to noise ratio) or PCA (ordering components in terms of variance)
2. Random initial selection of endmembers according to a predetermined number
3. Test each pixel in each endmember position and retain combinations with highest volume
gipsa-lab
Endmember extraction
gipsa-lab
Outlines
gipsa-lab
Linear spectral unmixing (LSU)
To be extimated
M
x = ai si + w
i=1
Known terms
Band b
The goal is to find extreme pixel vectors
(endmembers) that can be used to unmix other
mixed pixels in the data using a linear mixture
model.
Band a
gipsa-lab
Abundance estimation
gipsa-lab
Unconstrained abundance estimation
When all the endmember information (i.e., the number of endmembers and their
spectral signatures) are known, abundances can be estimated via the least
squares solution.
gipsa-lab
Non-negative constrained abundance estimation
gipsa-lab
Fully constrained abundance estimation
Actually, the sum-to-one constraint can be easily satisfied by adding a row vector
with all elements being one to the endmember matrix M, adding an element one
to the pixel vector r, and solving the resulting least squares problem.
gipsa-lab
Aviris Lunar Lake experiment
C V
S A
Five endmembers: Cinder (C), Playa lake (P), Rhyolite (R), Vegetation (V), Shade (S)
Six endmembers: Cinder (C), Playa lake (P), Rhyolite (R), Vegetation (V), Shade (S),
Anomaly (A)
Pixel reconstruction error =
gipsa-lab
Aviris Lunar Lake experiment
Shade
Vegeta<on
Abundance maps from unconstrained least squares (UCLS) linear unmixing
with five endmember (pixel reconstruction error = 8.85103)
gipsa-lab
Aviris Lunar Lake experiment
Shade
Vegeta<on
Abundance maps from non-negativity constrained least squares (NCLS) linear
unmixing with five endmember (pixel reconstruction error = 1.02104)
gipsa-lab
Aviris Lunar Lake experiment
Shade
Vegeta<on
Abundance maps from fully constrained least squares (FCLS) linear unmixing
with five endmember (pixel reconstruction error = 1.43104)
gipsa-lab
Aviris Lunar Lake experiment
gipsa-lab
Weighted Least Square
The WLS solution for the abundances of the known endmembers can be
estimated as:
gipsa-lab
Weighted Least Square
P1
P2
P3
P4
P5
Abundance maps from UCLS linear unmixing when only five panel signatures
are known
P1
P2
P3
P4
P5
Abundance maps from weighted least squares (WLS) linear unmixing when
only five panel signatures are known
gipsa-lab
Weighted Least Square
B
M
K
C
ve
known
endmembers:
alunite
(A),
buddingtonite
(B),
calcite
(C),
kaolinite
(K),
muscovite
(M)
gipsa-lab
Weighted Least Square
alunite
(A)
buddingtonite
(B)
calcite
(C)
kaolinite
(K)
muscovite
(M)
Abundance maps from UCLS linear unmixing when only five mineral signatures
are known
alunite
(A)
buddingtonite
(B)
calcite
(C)
kaolinite
(K)
muscovite
(M)
Abundance maps from weighted least squares (WLS) linear unmixing when
only five mineral signatures are known
gipsa-lab
remarks
gipsa-lab
Outlines
gipsa-lab
Nonlinear spectral unmixing
gipsa-lab
Linear vs nonlinear mixing
Linear or nonlinear?
gipsa-lab
Linear vs nonlinear mixing
50%-50%
gipsa-lab
Linear vs nonlinear mixing
Data cloud
shifted to low
albedo area
gipsa-lab
Linear vs nonlinear mixing
False
F endmembers!!
E
gipsa-lab
Linear vs nonlinear mixing
gipsa-lab
Standard NN based unmixing
Incorporates a learning process based on known, randomly selected training samples.
PCA, MNF, DWT, ICA,
Dimensionality reduction
Randomly selected
gipsa-lab
Standard NN based unmixing
Back-propagation learning from selected training samples.
The training stage is the most crucial: how many training samples are required?.
tl1 a1
1 1
1
tl2 M a2
2 2
M
M M M
tlN Hidden layer aE
N E
Input layer Output layer
gipsa-lab
Standard NN based unmixing
Combined linear/nonlinear architecture #1
Nonlinear refinement of linear abundance estimations (using FCLSU).
Limitation: this approach does not take into account the full spectral information.
Estimation of number
of endmembers, p
gipsa-lab
Standard NN based unmixing
Combined linear/nonlinear architecture #2
MLP initially trained using only p endmembers provided by an automatic algorithm.
Nonlinear refinement from a linear initial weight condition using training samples.
The full spectral information is used throughout the process.
Choice of endmember finding algorithms for initialization (AMEE used in this work).
Choice of intelligent training sample selection algorithms
Automatic endmember
extraction algorithm
gipsa-lab
Standard NN based unmixing
Training of neural network architecture
Need for algorithms able to automatically search for the most useful training samples.
Potential to direct ground-truth data collection to the most useful training sites.
In the linear mixture model, the most highly pure patterns are of interest.
In the nonlinear mixture model, border training patterns are required.
Extreme
Core
Band b
Border
Band b
Band a Band a
gipsa-lab
Standard NN based unmixing
p endmembers
gipsa-lab
Standard NN based unmixing
p endmembers
gipsa-lab
Mustards database with known proportions
Database of 26 pure and mixed (binary & ternary) spectra.
Collected using RELAB, a bidirectional spectrometer.
211 spectral bands in the range 0.4 2.5 m.
Ground-truth information about fractional abundances is available for each spectra.
gipsa-lab
Mustards database with known proportions
Pure signatures
Binary mixtures
Ternary mixtures
gipsa-lab
Outlines
gipsa-lab
Bi-linear spectral unmixing
bilinear models are defined by spectral components appearing in the widely used
LMM but also by bilinear terms corresponding to possible interactions between the
different materials of the scene.
M
x = ai si + w product of the ith and jth spectra
i=1
R R 1 R
m1,i m1, j
x = ar sr +
i, j mi m j + w
.
r =1 i=1 j =i+1
mi m j = .
amplitude of the interaction term due to the .
ith and jth components m m
L,i L, j
gipsa-lab
Bi-linear spectral unmixing
bilinear models are defined by spectral components appearing in the widely used
LMM but also by bilinear terms corresponding to possible interactions between the
different materials of the scene.
Nascimento model:
M m1,i m1, j
x = ai si + w .
i=1 mi m j = .
product of the ith and jth spectra
.
m m
R R 1 R
L,i L, j
x = ar sr + i, j mi m j + w
r =1 i=1 j =i+1 ar 0
i, j 0
R R 1 R
amplitude of the interaction term due to the
ith and jth components
a +
r i, j =1
r =1 i=1 j =1+1
gipsa-lab
Bi-linear spectral unmixing
R*
x = a*p m*p + w
p =1
Where:
gipsa-lab
Bi-linear spectral unmixing
Fans model:
R R 1 R
x = ar sr + a a m m
i j i j +w
r =1 i=1 j =i+1
The Fans model assumes that the amplitudes of the interactions depends on the
component fractions involved in the mixture.
gipsa-lab
Bi-linear spectral unmixing
Generalized model:
R R 1 R
x = ar sr + i, j ai a j mi m j + w
r =1 i=1 j =i+1
The generalized model assumes that the contribution of the interaction term
mi*mj is proportional to the fractions of the involved components with an
amplitude i,jaiaj
gipsa-lab
Summary
VD estimation
To estimate the number of endmembers to be extracted
Dimensionality reduction (optional)
To improve the performance of endmember extraction
Endmember extraction
To determine endmember signatures, which may or may not be image pixels
Endmember selection (optional)
To optimize the endmember set that is actually used for unmixing
Abundance estimation (linear or nonlinear)
To estimate endmember abundance, where linear and nonlinear unmixing may
be integrated.
gipsa-lab