UAV (Unmanned Aerial Vehicle)

Data Acquisition and Processing

Rongjun Qin

Part of the slides are adopted from R.Lathrop and

 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Aims and Goals

To have a closer look 3D data processing, taking

UAV images as an example
Not a hands on, but a step-by-step demonstration.
How the UAV product can be used for disaster
management.
What is the limitation of the UAV.
How much time and effort are expected by
performing a UAV mission.

2
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Software availability
Commercial Product:
INPHO Stuttgart
VITUROSO WuHan University
PixelGrid CASM (Chinese Academy of Surveying and Mapping)
Pixel4UAV- EPFL (cole polytechnique fdrale de Lausanne)
APS Menci software
Drone Mapper
LPS - Leica

Photomodeler, PhotoScan -- (close range)

Open Source:
Apero IGN
Bundler Noah Snavely (computer vision)
OpenMVG (computer vision)
ArcheOs
Not Inclusive !!! 3
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Software

Iwitness for camera calibration

Pix4UAV for Aerial triangulation and Orthophoto
generation
Quick Terrain Modeler for flood analysis.
SURE software for Image matching and point
clouds generation

Alternatives:

Open Source Solutions: Apero

SURE Software
DSM Generation. 4
Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

5
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

UAV take-off and landing

6
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Flight Planning parameters

Focal length: 16 mm

Pixel size: 5 microns

Image size: 4592 X 3056 -> 22.96 mm X 15.28 mm

Flying height: ? Higher fewer images, lower
resolution minimal height = = 160 m

Wanted GSD: 5 cm

Overlap: 80% forward, 60% side

7
Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

8
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Camera Pre-calibration
It is always good to calibrate your camera systematically, instead of the
replying
on self-calibration, for the reason:
Self-calibration is designed for compensate minor distortion induced by the
unpredictable parameters such as humidity and temperature of the air. It
might face risk of failure caused by unreliable network and over-
parameterization.
Different bundle system includes different self-calibration parameters, it may
not be sufficient enough to model the calibration parameters.
Signalized targets are more accurate to locate tie points.

9
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Aerial Triangulation Bundle Adjustment

Aerial triangulation is a process of determining the geo-location
and
orientation of the camera.

Input: Images, GPS observation

The use of the GPS observation can be used in different ways:

1) Purely for tie points matching, neighborhood Searching.
2) As the initial parameters for bundle solution.
3) As the observation for bundle solution.

10
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

An Alternative solution Open source

Apero software Arotriangulation Photogrammtrique

Exprimentale Relativement Oprationnelle

Windows, Linux, MacOS

11
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Open Source Solution with Apero-GUI

12
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Tie point extraction

GPS information is purely used for pair-wise tie points matching.
1) Automatic determination of the pair-wise matching
3 times average neighboring distance are used for searching
neighborhood.
2) Line based extraction mode
3) Multi-scale based exhaustive search.

13
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Demonstration of the Apero

RES:[IMG_5588.tif] ER2 0.124029 Nn 99.9769 Of 4335 Mul 0 Mul-NN 0 Time

0.160253

[IMG 5588.tif] is obviously the name of the image;

[ER2 0.143439] is the square root of the weighted average of quadratic
residuals;
[Nn 99.9769] is the percentage of residuals that are under EcartMax; it
should be over 95%, or
else it may signify that you have got residuals just because you have thrown
away high residuals!
Of 4327 is the number of tie points;
Mul 0 is the number of multiple points;
Mul-NN 0 is the number of multiple points having residuals under EcartMax;

14
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

DSM and Ortho-Image Generation

15
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Introduction to SGM matcher

Hierarchical Semi-global matching, developed by the University of Stuttgart

Original algorithm from Heiko Hirschmueller

Stereo matching: Computation of disparity map between rectified images

16
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Pixel-wise matching is not unique. Therefore add additional cost

(penalty) on the disparity steps (discontinuity):

(2)

17
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Hierarchical Semi-global Matching

Traditional Semi-Global matching: require large memory storage,

computation time on normal computer. For UAV images,
the disparity level easily goes up to more than 1000, which
require memory for M X N X D for computation.

Wenzel et al: Perform it hierarchically, from a coarse level to a

finer level, the disparity searching ranges comes from the
former level.

18
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

Pixel-wise matching is not unique. Therefore add additional cost

(penalty) on the disparity steps (discontinuity):

(2)

19
 Department of Civil, Environmental and Geodetic Engineering
Department of Electrical and Computer Engineering

It is a stereo-matching algorithm, point clouds are generated

for every stereo model, the result of the multiple models
are fused as a post step.

Multi-stereo Processor Demonstration

20