Technical exploitation needs for defence applications: SAR imagery

Image Analysis Laboratory, Observation Systems Area Dept. of Earth Observation, Remote Sensing and Atmosphere (DOTTA) INTA

Severino Fernndez fdezas@inta.es Support tools for IMINT exploitation Patrick Vaughan vaughanmp@inta.es SIIRS and reference manual Enrique Nicols nicolasge@inta.es SAR geometric exploitation

Presented at the Congress on technologies and applications of synthetic aperture radar (SAR), 30th-31st May, 2011, INTA, Torrejn de Ardoz, Spain.
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Objective:

Support tools for IMINT exploitation

Identify IMINT user needs for SAR image exploitation in the spanish MoD Identify techniques considering availability of Helios I (since 1995) and Helios II (since 2005) imagery in archive, and fast access Helios II imagery (Ground Segment in Torrejon)

Consider availability of Pleiades imagery in archive (Pleiades launch planned for beginning 2012) and fast access to Pleiades 1 imagery (Ground Segment in Torrejon).
Identify modern methods that can improve IMINT analyst productivity and confidence on interpretation results.

Build prototype to assess feasibility characterize results (for which cases produces reliable results, for which cases does not) once robustness verified, evaluate operational feasibility (processing time) if both OK, integrate into IMINT user workstation
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

IMINT activity/need Complementary use of large asset and eventual shortly simultaneously acquired optical high resolution imagery Detect: Ships: detection seems to be feasible already Ground vehicles: needs characterization Paked aircraft: detection seems feasible (somehow like ships, background is simple) Identify vehicles Difficult with 1 m resolution Methods must produce results in seconds for images of size of workstation screen (somehow 1000x1000 pixels)
OSA May 30-31, 2011

Experiment-Integration automatic or largely automatic image coregistration of SAR and optical imagery. SAR/optical image data fusion after coregistration Evaluate visual interpretability enhancement with Superresolution methods Evaluate detectability with same or similar methods as for ships Evaluate subaperture processing for target enhancement Evaluate identification with feature classification+feature enhancement with Superresolution methods Once robustnessis validated, optimize code Multithreading implementation for multicore workstations

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Imagery co-registration. Preliminary results and status.
Mutual information similarity measure Simple local deformation (e.g. translation), global deformation derived from local one by interpolation Optimization: probably stochastic gradient Several open source libraries available Some provide robust optimization with adaptive parameters, but not yet refactored for multithread Others already optimized some similarity measures, some optimizers and natively multithreaded Has been just preliminary validated (suspended because no resources allocated)

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Imagery co-registration. Some results with real data (Ikonos+Terrasar Torrejon Air Base) Automatic co-registration with local displacement and global spline deformation. Pending Actions:
Exhaustive tests to verify robustness of approach Characterize at least following aspects: Behaviour on buildings, infrastructure Timing, maximum size with acceptable processing time using multicore PC Identify failure cases

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Before

Support tools for IMINT exploitation

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

After

Support tools for IMINT exploitation

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

SAR imagery superresolution Concept demonstration prototypes:
Adaptive Weighted Norm (spectrum) Extrapolation 2D version coded AR spectrum estimation a particular implementation coded APES (Amplitude and Phase Estimation) and Minimum Variance (Capon) open source code from Prof. Erik Larsson tested with available imagery
Limited results due to lack of resources.
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

SAR imagery superresolution- Adaptive Weighted Norm Extrapolation:
Minimum wighted norm, matches given data and frequency distribution.

Test: A subaperture is taken from the data and extrapolated to compare with the original image.

Original image

image from the lower frequency components

image from spectrum extrapolation

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

SAR imagery superresolution - Adaptive Weighted Norm Extrapolation Extrapolation of all data of spectrum. No comparison with reference possible. Note the apparent better resolution.
These are not part of aircraft

Original image

image from spectrum extrapolation

Optical image (Ikonos)

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

SAR imagery superresolution - 2-D AR extrapolation Two quadrant AR coefficients are computed from the data (the signal history obtained from the image). The new signal history is obtained from the AR coefficients.

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Prof. Larsson implementation of APES and Capon AWNE seems visually more attractive. But both APES and Capon perform better for single scatterers.
AWNE

Original
OSA May 30-31, 2011

Capon

APES (Amplitude and Phase)

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Prof. Larsson implementation of APES and Capon Using an ideal signal produced by three scatterers, APES and Capon reconstruct the image with more fidelity than AWNE. The ideal scatterers are correctly situated. .

AWNE

Signal of 3 scatterers
OSA May 30-31, 2011

DFT

APES
Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Present state:
The usefulness for support of visual image interpretation of the various methods for SR from single images seems very doubtful . At least for the resolution of Paz (TerrasarX). Several authors state that these methods improve the results of automatic target recognition.

We would have to check this on Paz (TerrasarX) data. This would imply implementing a target recognition prototype. Another question arises: do these methods have any influence on the results of automatic co-registration of SAR and optical data?
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Ground target detection Simple approach using global-local (CFAR) detection with Weibull and K distributions + morphology operation on results

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Detection of faint tracks in coherence image. Denoising.

Method based on Poisson statistics for pixel intensity (from fluorescence imagery application). Visibility improvement for extended regions. can we apparently see features in noisiy image just because we see more in denoised one?
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Support tools for IMINT exploitation

Thank you

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Background

SIIRS (SAR Image Interpretability Rating Scale) & Reference Manual
Scope: military applications operational SAR image analysis Exploitation need: to provide operational users with answers to: a) PAZ imagery request parameters, e.g.: What image mode ? What polarisation ? How many images do I need ? b) PAZ Capability & task planning, e.g.: What image processing ? What targets can I detect, recognise or identify? Expected time frame?
Solution: establish straightforward reference guidelines per mission type, based on reference documents: NIIRS and STANAG 3596 Framework: Strategic Plan of OSA-DOTTA (INTA) Milestones: 3+ (2011-2013+) Platform and outsourcing: TBD (Industry, tech specs.)

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Outline and workflow

SIIRS (SAR Image Interpretability Rating Scale) & Reference Manual
NIIRS
Interpretability (analysis capability) Per spatial resolution Per wavelength range (VIS,NIR, MIR,SAR)
Per target examples

Target categories Reporting fields Recce requesting reporting guide USER FEEDBACK

ADAPT EXPAND

SIIRS (phase 1)

TOOLS AND ALGORYTHMS

VALIDATE FIELD TESTING

Reference Manual (phase 2)

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Basic contents

SIIRS Reference Manual
How to do it: Data requirements:
Number of images & acquisition modes

What can we discriminate:

List of targets per STANAG 3596 category that can be: Detected Distinguished Recognised Identified

spatial & time resolution Recommended polarisation Estimated time (acquisition + processing)

Level of certainty/feasibility Processing methodology Alternative methodology

Processing tool requirements Level of complexity

Examples

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Case Scenario

SIIRS
Recce request for: Target CAT 010B+2b. Occupation: disturbed earth

Reference Manual
Do we have the minimum requirements? Any examples ?

CAT 010B+2b. Is this included in the list ?

Disturbed earth detection using Coherent Change Detection (CCD)

Low end

High end

OSA May 30-31, 2011

VALIDATION REQUIRED?
Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: CCD examples

Google Earth TerraSAR-X SM ~1.5m pixel 11 day CCD CCD
Airborne SAR-X ~25cm resolution

2nd date (I) image

Preiss and Stacey, DSTO 2008

?
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Validation & Fieldwork

REAL SCENARIO CAMPAIGN 2009 COMING UP...

Airborne SAR X (1m pixel) from Radar Laboratory (Dpt. Of Radiofrequencies and Electronic Technology-INTA)
Validation and information on: Signal penetration on cammo nets Target discrimination vs. clustering Target size and pose vs. detection Environment /clutter interference

Validation studies of Tandem-X & TerraSAR-X Data applications for security and defence
DLR Scientific Project Proposal

Validation aims:
Pointing & DEM accuracy Pixel X,Y,Z location accuracy Trafficability Disturbed earth & CCD

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SIIRS & Reference Manual: Summary

NIIRS VALIDATION & FIELDWORK

BENCHMARKING & STATE OF THE ART

Funnel & filter

SIIRS

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Thank you very much !

Patrick J. Vaughan Martn-Mateo

vaughanmp@inta.es

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Objective:
Identify IMINT user needs for SAR image geometric exploitation in the Spanish MoD.
Analyze PAZ satellite capabilities to cover the user needs. Training with TerraSAR-X and TanDEM-X.

Analyze the different techniques for DEM generation from SAR images: Cross-track single pass interferometry. Cross-track repeat pass interferometry. Repeat pass radargrammetry. Define a methodology to characterize the influence of the observation geometry in: the location accuracy of the SAR images the DEM height accuracy generated.
Define a ground truth area for geometric CAL/VAL of SAR/Optical images and other applications.

User support for baseline tasks concerning orthorectification, virtual scenarios generation and other applications.
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

IMINT activity/need
-Techniques to generate altimetric information to use as input in other applications. - Location/height accuracy knowledge

Experiment-Integration
Effects on data due to variations in geometric conditions. To learn geometric behaviour in TSX and TDX for the future PAZ satellite.

Produce: DSMs (Digital Surface Models). DTMs (Digital Terrain Models). GDMs (Ground Deformation Models). GCPs (Ground Control Points). Course training material. To: Block adjustment. Orthorectify. Virtual scenarios generation. Underground structure detection. 3D feature extraction. User training. Other.

Analyze the influence of parameters on the pointing and height accuracy: the post-processed orbit type. processed product type. elevation beam angle. baseline size. orbit direction. polarization. single pass vs. repeat pass. influence of software processing algorithms. Characterize the differences between techniques for DEM generation: Limitations. Best choice. Effort in the generation.
Define protocols and training for users.
Technical exploitation needs for defence applications: SAR imagery

Define protocols and training for users.

OSA May 30-31, 2011

SAR Geometric Exploitation

Experiments: Validation studies of TanDEM-X data applications for security and defence (proposal submitted to the DLR)
- Test site experiments (TBC): -Image Pairs (TSX/TDX): -3 SL mode. -3 HS mode. -5 HS300 mode. -10 SM mode. -1 SC mode. - Interferometric single pass: -23DEMs. -Interferometric repeat pass: -36DEMs. -Radargrammetric repeat pass: -55DEMs. -CCD analysis: 36 products. -Border control. -Trafficability. -Disturbed soil. -DEM fusion techniques. -SAR focussing.
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- Test site definition (TSAs): - Terrain type scenarios: Flat area, smooth/medium relief, high relief.
- Cover type scenario: bared soil, shrubland, forest, manmade.

25x30km

52x30km

10x10km OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- Planimetric accuracy control: - (SAR) Corner reflectors: trihedral / offset trihedral / luneberg lenses (TBC).

Trihedral

Offset Trihedral

Luneberg Lenses
OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth works:
- Planimetric accuracy control: - (SAR) Corner reflectors: trihedral / offset trihedral / luneberg lenses (TBC). - (SAR) Identified Permanent Scatterer Points and Intensity Change Points (IPSPs - ICPs)

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth works:
- Planimetric accuracy control: - (SAR) Corner reflectors: trihedral / offset trihedral / luneberg lenses (TBC). - (SAR) Identified Permanent Scatterer Points and Intensity Change Points (IPSPs - ICPs)
Correct match: Street lamp base Measurable IPSP No match: Probably a tennis umpire chair

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- Planimetric accuracy control: - (SAR) Corner reflectors: trihedral / offset trihedral / luneberg lenses (TBC). - (SAR) Identified Permanent Scatterer Points and intensity change points (IPSPs - ICPs). - (Optical) GPS Rapid Static observations (GCPs grid).

- (Optical) Spanish geodetic reference network vertex (REGENTE, ROI).

Legend
GCPs_grid TSAs REGENTE vertex ROI vertex GCPs observed

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- DEM accuracy control: - Continuous: - Other well-known DEM sources. (LiDAR, airborne interferometry, stereo-photogrammetric, spaceborne interferometry. -QA of the continuous sources with discrete data will be made. ASTER-GDEM SRTM

DTED2

IGN25

Intermap NEXTmap

And others: PNOA, LiDAR spots

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- DEM accuracy control:
Continuous: - Other well-known DEM sources. (LiDAR, airborne interferometry, stereophotogrammetric, spaceborne interferometry)

Discrete: - GCPs. - Spanish geodetic reference network vertex and marks (REGENTE, ROI, REDNAP). - DGPS Z-tracks. - Vehicle tracks. - Walk tracks. - Slope tracks. - Tree height measurement.

Legend
GCPs_grid TSAs REGENTE vertex ROI vertex GCPs observed Z_TRACKS REDNAP XY PRECISION_m 0,05 0,1 1

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

SAR Geometric Exploitation

Ground truth work:
- DEM accuracy control:
Continuous: - Other well-known DEM sources. (LiDAR, airborne interferometry, stereophotogrammetric, spaceborne interferometry)

Discrete: - GCPs. - Spanish geodetic reference network vertex and marks (REGENTE, ROI, REDNAP). - DGPS Z-tracks. - Vehicle tracks. - Walk tracks. - Slope tracks. - Tree height measurement.

Legend
GCPs_grid TSAs REGENTE vertex ROI vertex GCPs observed Z_TRACKS REDNAP XY PRECISION_m 0,05 0,1 1

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery

Thanks for your attention.

Severino Fernndez fdezas@inta.es Support tools for IMINT exploitation Patrick Vaughan vaughanmp@inta.es SIIRS and reference manual Enrique Nicols nicolasge@inta.es SAR geometric exploitation

OSA May 30-31, 2011

Technical exploitation needs for defence applications: SAR imagery