Route Planning with FOSS: OpenStreetMap, PostGIS, OSRM and OpenLayers

Michael Regg
Institute For Software University of Applied Sciences Rapperswil

/ch/open Workshop, Zrich November, 2012

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

2 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

3 / 40

What is GIS?
GIS stands for Geographic Information System Used to answer what is where on the Earths surface GIS data is typically stored in a Spatial Database A spatial database
denes special data types for geometric objects (e. g., polygons) allows to save geometric data in regular (relational) databases provides special functions and indexes for querying and manupulating spatial data

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

4 / 40

Map projections & spatial reference systems

Basic problem: How to get from a round world to a at map? Map projections: portray Earths curved to a at surface

Spatial Reference System (SRS) European Petrolueum Survey Group Geodesy (EPSG) Geodesy is the study of the shape of the Earth WGS84: The World Geodetic System is a standard for use in cartography, geodesy and navigation CHLV03: The coordinate system for Switzerland
Michael Regg (IFS) Route Planning with FOSS November, 2012 5 / 40

Layer concept
GIS is closely associated with the layer concept A GIS allows to work in various levels and their proper representations Goal: Combination of data to get new information

Figure: http://www.digital-geography.com/concept-of-layers-in-a-gis
Michael Regg (IFS) Route Planning with FOSS November, 2012 6 / 40

GIS le formats
Standards for encoding geographical information into a le Raster vs. vector formats Common vector formats:
Drawing Interchange Format (DXF): CAD data le format developed by Autodesk ESRI Shapele: popular geospatial vector data format from Environmental Systems Research Institute (ESRI) Keyhole Markup Language (KML): XML dialect for expressing geographic visualization within two-dimensional maps and three-dimensional Earth browsers Spatialite: free GIS that enables spatial queries and objects for SQLite

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

7 / 40

Geometric data types

A feature represents the basic data unit in a GIS How can we store and represent geographic features in a database? Simple Feature for SQL (SFS) Model Storage model for geographical data using well-known text (WKT) Basic geometric data types: points, linestrings, polygons

Figure: Source Wikipedia

Michael Regg (IFS) Route Planning with FOSS November, 2012 8 / 40

Overview SFS data types

Figure: Source Wikipedia

Michael Regg (IFS) Route Planning with FOSS November, 2012 9 / 40

Well-known text (WKT)

Text markup language for representing vector geometry objects on a map Examples (Source Wikipedia): Type Point WKT POINT (30 10) Figure

LineString LINESTRING (30 10, 10 30, 40 40) Polygon Polygon with hole
Michael Regg (IFS)

POLYGON ((30 10, 10 20, 20 40, 40 40, 30 10)) POLYGON ((35 10, 10 20, 15 40, 45 45, 35 10), (20 30, 35 35, 30 20, 20 30))
Route Planning with FOSS November, 2012 10 / 40

Spatial Relationships
An important goal when using a GIS is determining the spatial relationships between geometric objects

Figure: Source Wikipedia

Michael Regg (IFS) Route Planning with FOSS November, 2012 11 / 40

Spatial Indices
A spatial index is used to optimize spatial queries Spatial order structure: What is spatially close together should also be close in memory

Figure: R-Tree example

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

12 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

13 / 40

What is PostGIS?
Extension of PostgreSQL, spatially enables the PostgreSQL server Implements the SFS model Open Source (GNU General Public License) Data loaders (e. g., ESRI shape les) >330 geometric functions: distance, area, union, intersection, etc.

Latest release is 2.0.1 (June 2012)

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

14 / 40

How to create PostGIS enabled databases

Create PostGIS database template:
$ createdb template_postgis $ psql template_postgis -c "create extension postgis" $ psql template_postgis -f <installdir>/postgresql/share/contrib/ postgis-2.0/legacy.sql

Now we can create a DB from our template:

$ createdb -T template_postgis gis_db

To verify the installed PostGIS version, use this:

$ psql gis_db -c "SELECT PostGIS_Full_Version()"

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

15 / 40

PostGIS Metadata tables

There are two OpenGIS meta-data tables SPATIAL_REF_SYS has >3000 spatial reference systems:
CREATE TABLE srid auth_name auth_srid srtext proj4text spatial_ref_sys ( INTEGER NOT NULL PRIMARY KEY, VARCHAR(256), INTEGER, VARCHAR(2048), VARCHAR(2048));

GEOMETRY_COLUMNS contains the spatial columns:

CREATE TABLE geometry_columns ( f_geometry_column VARCHAR(256) NOT NULL, coord_dimension INTEGER NOT NULL, srid INTEGER NOT NULL, type VARCHAR(30) NOT NULL);

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

16 / 40

Creating basic geometries with PostGIS

We create a basic point object (x,y):
SELECT ST_Point(1, 3) AS ExamplePoint;

No spatial reference system used (default is Cartesian grid). Lets use WGS84:
SELECT ST_SetSRID(ST_Point(8.515342,47.389058), 4326);

The same can be done by using WKT:

SELECT ST_GeomFromText(POINT(8.515342,47.389058), 4326);

The same for LineStrings and Polygons:

SELECT ST_GeomFromText(LINESTRING(1 1,2 2,3 3)) AS MyLine; SELECT ST_GeomFromText(POLYGON((0 1,1 -1,-1 -1,0 1))) As MyTriangle;
Michael Regg (IFS) Route Planning with FOSS November, 2012 17 / 40

Creating a spatial table

1

Create a normal non-spatial table:

CREATE TABLE beers ( id NOT NULL PRIMARY KEY, name VARCHAR, price FLOAT4);

Add a spatial column:

SELECT AddGeometryColumn(public, beers, geom, 4326, POINT, 2);

Syntax of AddGeometryColumn:
AddGeometryColumn(<schema_name>, <table_name>, <column_name>, <srid>, <type>, <dimension>)
Michael Regg (IFS) Route Planning with FOSS November, 2012 18 / 40

Creating a spatial index

Create a spatial index:
CREATE INDEX in_beers ON beers USING gist(geom);

New dataset for PostGIS functions:

INSERT INTO beers(name, price, geom) VALUES ( Manor, 1.15, GeometryFromText(POINT(8.81915 47.22640),4326)); SELECT * from beers order by price; 21;"Coop Sunnehof"; 1.05;"01010000..." 18;"Manor"; 1.15;"01010000..." 20;"Dorflade"; 2.4; "01010000..." 23;"Rappi Bier Factory";2.9; "01010000..." 22;"Lido Markt"; 3.1; "01010000..." 19;"Biergarten"; 3.5; "01010000..." 17;"Nelson Pub"; 4.5; "01010000..."
Michael Regg (IFS) Route Planning with FOSS November, 2012 19 / 40

PostGIS Functions
SELECT name, price, round( st_distance_sphere(geom, GeometryFromText(POINT(8.816511 47.223064), 4326) )::numeric, 0) AS "distance" FROM beers ORDER BY distance; "Biergarten"; 3.5; 235 "Dorflade"; 2.4; 300 "Nelson Pub"; 4.5; 381 "Manor"; 1.15;422 "Coop Sunnehof";1.05;706

Note that we must convert the point to EPSG 4326! ST_Distance_Sphere: Returns minimum distance in meters between two lon/lat geometries
float ST_Distance_Sphere(geometry lonlatA,geometry lonlatB)
Michael Regg (IFS) Route Planning with FOSS November, 2012 20 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

21 / 40

OpenStreetMap Data
OpenStreetMap (OSM) export format is a XML format Topological data structure with four data types:
Nodes are points with a geographic position, stored as WGS84 coordinates (e. g. POIs) Ways are ordered node lists representing a polyline (e. g. streets, rivers, parks) Tags are used to store metadata about the map objects Relations are used for representing the relationship of existing nodes and ways (e. g. turn restrictions on roads)

OpenStreetMap data: www.geofabrik.de OSM data for Switzerland in PBF Format (Protocolbuffer Binary Format): download.geofabrik.de/osm/ europe/switzerland.osm.pbf
Michael Regg (IFS) Route Planning with FOSS November, 2012 22 / 40

Load OSM data into PostgreSQL

osm2pgsql is a utility that converts OSM data to PostgreSQL databases osm2pgsql is a lossy conversion utility Two main modes of running: normal and slim mode Normal mode uses memory for intermedia storage Slim mode uses 3 on-disk tracking tables: planet_osm_nodes, planet_osm_ways, planet_osm_rels

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

23 / 40

Load OSM data into PostgreSQL

$ osm2pgsql --create --slim --extra-attributes \ --database <DB_NAME> --prefix osm --style \ /usr/local/share/osm2pgsql/xyz.style \ --username <DB_USER> --port <DB_PORT> \ --hstore-all --input-reader pbf \ --extra-attributes switzerland.osm.pbf

-hstore: Generate a hstore (key/value) column By using hstore we can use any tag in SQL queries:
gis_db=> SELECT COUNT(*) FROM planet_osm_point WHERE ((tags->man_made) = tower);

osm2pgsql overwrites the tables and create them fresh by default (use -append instead if necessary)

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

24 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

25 / 40

Introduction
High performance routing machine Does not use an A* variant to compute shortest path, but Contraction Hierarchies (CH) Written in C++, available under the GNU license

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

26 / 40

Installation
For Ubuntu 12.04:
$ sudo apt-get install build-essential git scons \ pkg-config libprotoc-dev libprotobuf7 \ protobuf-compiler libprotobuf-dev libosmpbf-dev \ libpng-dev libbz2-dev libstxxl-dev libstxxl-doc \ libstxxl1 libxml2-dev libzip-dev \ libboost-thread-dev libboost-system-dev \ libboost-regex-dev libboost-filesystem-dev \ lua5.1 liblua5.1-0-dev libluabind-dev $ git clone https://github.com/DennisOSRM/Project-OSRM.git $ scons -j4

More information: https://github.com/DennisOSRM/ Project-OSRM/wiki/Building%20OSRM

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

27 / 40

Conguration
server.ini

server.ini contains properties for the routing engine:

Threads = 8 IP = 0.0.0.0 Port = 5000 hsgrData=switzerland.osrm.hsgr nodesData=switzerland.osrm.nodes edgesData=switzerland.osrm.edges ramIndex=switzerland.osrm.ramIndex fileIndex=switzerland.osrm.fileIndex namesData=switzerland.osrm.names

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

28 / 40

Conguration
Routing proles

Routing proles are specied in Lua les Example car.lua:

peed_profile = { ["motorway"] = 90, ["motorway_link"] = 75, ["trunk"] = 85, ["trunk_link"] = 70, ["primary"] = 65, ["primary_link"] = 60, ["secondary"] = 55, ["tertiary"] = 40, ["residential"] = 25, ["living_street"] = 10, ["service"] = 15, ["ferry"] = 5, ["default"] = 50 }
Michael Regg (IFS) Route Planning with FOSS November, 2012 29 / 40

Running OSRM
Extracting the Road Network 1/2

OSM data contains information irrelevant to routing OSM data needs to be normalized before being processed This is done by the OSRM tool named extractor Parses the content of the OSM le and writes:
A .osrm le containing the routing data A .osrm.restrictions le containing the restrictions to make certain turns during navigation A .osrm.names le which contains the road names

Lets execute it:

$ ./osrm-extract switzerland.pbf

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

30 / 40

Running OSRM
Extracting the Road Network 2/2

The extractor is able to handle bzip2 compressed les as well as PBF les External memory accesses are handles by the stxxl library stxxl must be congured in a le .stxxl File format: disk=full_disk_filename,capacity,access_method Example: disk=/tmp/stxxl,25000,syscall

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

31 / 40

Running OSRM
Creating the hierachy and run OSRM

To create hierarchy (the precomputed data that enables OSRM to nd the shortest path extermly fast), type:
$ ./osrm-prepare switzerland.osrm \ switzerland.osrm.restrictions

Nearest-neighbor data structure and node map are created Afterwards, 4 les should exist:
map.osrm.hsgr (the hierarchy) map.osrm.nodes (the nodemap) map.osrm.ramIndex (stage 1 index) map.osrm.fileIndex (stage 2 index)

To run the engine, just type:

./osrm-routed
Michael Regg (IFS) Route Planning with FOSS November, 2012 32 / 40

OSRM Services
locate: Location of nearest node to a given coordinate nearest: Location of nearest point on any street segment for a given coordinate viaroute: Computation of shortest path between two coordinates given an ordered list of via points Query format:
http://server:5000/_service_?param1=value&...&paramX=value

Example (from here to Zrich Mainstation):

$ curl "server:5000/viaroute?loc=47.389058,\ > 8.515342&loc=47.378188,8.539204" | python -mjson.tool { "version": 0.3,"status":0,"status_message": "Found route between points","route_geometry": ["4","Duttweilerstrasse",239,4,17,"239m","NE",38], ["2","Foerrlibuckstrasse",363,7,27,"363m","E",84], "route_summary":{"total_distance":3220,"total_time":237,...
Michael Regg (IFS) Route Planning with FOSS November, 2012 33 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

34 / 40

Introduction
OpenLayers is a JavaScript library for displaying map data in web browsers (no server-side dependencies) Most popular open source software web-mapping client Ability to overlay proprietary non-OGC-compliant mapping layers with OGC WMS, WFS and WFS-T layers Various controls to build custom toolbars, menus and widgets for editing the map

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

35 / 40

Example
<script src="OpenLayers.js"></script> <script type="text/javascript"> var map; function init() { map = new OpenLayers.Map(map, { controls: [ new OpenLayers.Control.Navigation(), new OpenLayers.Control.PanZoomBar(), new OpenLayers.Control.LayerSwitcher({ascending:false}), new OpenLayers.Control.Permalink(), new OpenLayers.Control.ScaleLine(), new OpenLayers.Control.Permalink(permalink), new OpenLayers.Control.MousePosition(), new OpenLayers.Control.OverviewMap(), new OpenLayers.Control.KeyboardDefaults()], numZoomLevels: 6 }); map.addLayer(new OpenLayers.Layer.Google("Google Hybrid", { type : google.maps.MapTypeId.HYBRID, numZoomLevels : 20 })); var position = new OpenLayers.LonLat(lon, lat).transform(fromProjection, toProjection); var markers = new OpenLayers.Layer.Markers("Markers"); map.addLayer(markers); markers.addMarker(new OpenLayers.Marker(position)); } </script>

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

36 / 40

Outline
1

GIS 101 PostGIS OpenStreetMap Open Source Routing Machine OpenLayers Showcase: Tourpl

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

37 / 40

Tourpl

Tourpl Der Tourenplaner A route planning web application developed at University of Applied Sciences Rapperswil Travelling Salesman Problem (TSP) Uses PostGIS, OSM data, osm2pgsql, OSRM and OpenLayers Available at www.tourpl.ch Demonstration

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

38 / 40

Thanks for your attention!

IFS

INSTITUTE FOR SOFTWARE

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

39 / 40

Bibliography I
Regina O. Obe and Leo S. Hsu PostGIS in Action. Manning, 2011 Stefan Keller Unterrichtsmaterialien Geodatenbanksysteme. University of Applied Sciences Rapperswil, 2012 Dennis Luxen OSRM Wiki. https: //github.com/DennisOSRM/Project-OSRM/wiki

Michael Regg (IFS)

Route Planning with FOSS

November, 2012

40 / 40