Object Recognition in 3D scans

Bastian Steder
University of Freiburg, Germany
Willow Garage Intern Presentation 09/23/2010

Object Recognition in 3D scans

Goal: Object detection and

localization in point clouds
Bastian Steder - Object Recognition in 3D scans - intern presentation

Working on Range images

Main advantage:
Encodes negative information
Bastian Steder - Object Recognition in 3D scans - intern presentation

General Approach
Match features between model and scene
Estimate object candidate poses
Score and filter candidates

Bastian Steder - Object Recognition in 3D scans - intern presentation

Feature extraction
Mainly two steps:
Interest point detection:
Where should features be extracted
Calculation of a descriptor:
Describe the area around the interest point in a
way that makes it easy to compare

Bastian Steder - Object Recognition in 3D scans - intern presentation

Desired properties of our Features

Use positions where the surface is stable,
but with significant changes in surrounding (I)
Encode the surface structure (I+D)
Encode outer shape (I+D)
Provide local 6DOF transformation

=> NARF: Normal aligned radial features

Bastian Steder - Object Recognition in 3D scans - intern presentation

Border information
Three kinds of border points
Obstacle borders
Shadow borders
Veil points

Bastian Steder - Object Recognition in 3D scans - intern presentation

How to detect borders?

Possibilities:
Impact angles of the sensor beams
Changes in normals

What actually works well:

Changes in distances to
neighboring points

Bastian Steder - Object Recognition in 3D scans - intern presentation

Border extraction video

Bastian Steder - Object Recognition in 3D scans - intern presentation

Interest points extraction

Calculate a score for each image point
How much do the dominant directions of the
surface changes in the vicinity vary?
(border information & principal curvature)
How stable is the surface on the point itself?

Find maximums above a threshold

Bastian Steder - Object Recognition in 3D scans - intern presentation

10

NARF Interest point extraction video

Bastian Steder - Object Recognition in 3D scans - intern presentation

11

NARF descriptor video

Bastian Steder - Object Recognition in 3D scans - intern presentation

12

Results

ROC curve

Bastian Steder - Object Recognition in 3D scans - intern presentation

13

Example Step 1

Bastian Steder - Object Recognition in 3D scans - intern presentation

14

Spatial validation Validation points

Validation points on the model views for a fast spatial validation

Bastian Steder - Object Recognition in 3D scans - intern presentation

15

Example Step 2

Bastian Steder - Object Recognition in 3D scans - intern presentation

16

Overlapping poses + ICP

Removal of multiple instances of the same
object in the same position, coming from
different feature matches
Refinement of the object poses using ICP

Bastian Steder - Object Recognition in 3D scans - intern presentation

17

Example Step 3

Bastian Steder - Object Recognition in 3D scans - intern presentation

18

Spatial validation View Simulation

Comparison of what was actually seen and

what it should look like according to the model

Bastian Steder - Object Recognition in 3D scans - intern presentation

19

Example Step 4

Bastian Steder - Object Recognition in 3D scans - intern presentation

20

Video

Bastian Steder - Object Recognition in 3D scans - intern presentation

21

Final scene

Bastian Steder - Object Recognition in 3D scans - intern presentation

22

Software and Documentation

Currently some parts in PCL and some in
point_cloud_perception_experimental
Range image class, border extraction, NARF
interest point extraction, NARF descriptor,
model class, spatial verification, visualization
tools

Some Tutorials already exists

there will be more
Paper submitted to ICRA11

Bastian Steder - Object Recognition in 3D scans - intern presentation

23

Thank you!
Willow Garage in general for making this possible
Radu for always being available for (sometimes
stupid) questions and discussions
Kurt for giving valuable input
The other interns and normal employees for
making it a great time
The people in the kitchen for protecting me from
months of Fast Food

Bastian Steder - Object Recognition in 3D scans - intern presentation

24

Questions?

Bastian Steder - Object Recognition in 3D scans - intern presentation

25