User:G0ldfish

All my contributions to OpenStreetMap are released into the public domain. This applies worldwide. In case this is not legally possible, I grant anyone the right to use my contributions for any purpose, without any conditions, unless such conditions are required by law.

Osmium and QGIS

Osmium looks much like the tool I have always been waiting for to create shapefiles from OSM data for use in QGIS (and create scalable maps from it which is my main aim) without the need to set up a database to access with QGIS.

Conversion with Osmium

For the first part you need Linux. Not much of a problem nowadays though. Providing you have enough RAM and free disk space, you can install a VM player like VirtualBox and install e.g. Ubuntu or Debian into it.

The installation of Osmium following the description in the wiki is straightforward with most versions tested. You can try to build the examples after installing, but to me they did not mean much (maybe meant for those who intend to use Osmium as library in own software projects rather).

Instead, change to the osmjs subdirectory and build osmjs:

make osmjs

will create a new executable file named osmjs.

Now we need osm data and a configuration file to test. My configuration file currently looks like this:

var shp_pois = Osmium.Output.Shapefile.open('pois', 'point');
shp_pois.add_field('id', 'integer', 10);
shp_pois.add_field('type', 'string', 32);
shp_pois.add_field('name', 'string', 32);
 
var shp_areapois = Osmium.Output.Shapefile.open('areapois', 'polygon');
shp_areapois.add_field('id', 'integer', 10);
shp_areapois.add_field('type', 'string', 32);
shp_areapois.add_field('name', 'string', 32);
 
var shp_places = Osmium.Output.Shapefile.open('places', 'point');
shp_places.add_field('id', 'integer', 10);
shp_places.add_field('type', 'string', 32);
shp_places.add_field('label', 'string', 32);
shp_places.add_field('size', 'integer', 3);
 
var shp_tunnels = Osmium.Output.Shapefile.open('tunnels', 'line');
shp_tunnels.add_field('id', 'integer', 10);
shp_tunnels.add_field('type', 'string', 32);
shp_tunnels.add_field('level', 'integer', 3);
shp_tunnels.add_field('ref', 'string', 32);
shp_tunnels.add_field('grade', 'integer', 1);
 
var shp_bridges = Osmium.Output.Shapefile.open('bridges', 'line');
shp_bridges.add_field('id', 'integer', 10);
shp_bridges.add_field('type', 'string', 32);
shp_bridges.add_field('level', 'integer', 3);
shp_bridges.add_field('ref', 'string', 32);
shp_bridges.add_field('grade', 'integer', 1);
 
var shp_roads = Osmium.Output.Shapefile.open('roads', 'line');
shp_roads.add_field('id', 'integer', 10);
shp_roads.add_field('type', 'string', 32);
shp_roads.add_field('ref', 'string', 32);
shp_roads.add_field('grade', 'integer', 1);
 
var shp_lines = Osmium.Output.Shapefile.open('lines', 'line');
shp_lines.add_field('id', 'integer', 10);
shp_lines.add_field('type', 'string', 32);
 
var shp_background = Osmium.Output.Shapefile.open('background', 'polygon');
shp_background.add_field('id', 'integer', 10);
shp_background.add_field('type', 'string', 32);
 
Osmium.Callbacks.node = function() {
	if (this.tags.amenity) {
		shp_pois.add(this.geom, {
			id: this.id,
			type: this.tags.amenity,
			name: this.tags.name
		});
	} else if (this.tags.place && this.tags.place.match(/^(city|town|village|hamlet|suburb)$/)) {
		var size;
		if (this.tags.place == 'city') {
			size = 144;
		} else if (this.tags.place == 'town') {
			size = 120;
		} else if (this.tags.place == 'village') {
			size = 96;
		} else {
			size = 60;
		}
		shp_places.add(this.geom, {
			id: this.id,
			type: this.tags.place,
			label: this.tags.name,
			size: size
		});
	}
}
 
Osmium.Callbacks.way = function() {
	if (this.tags.highway) {
		var grade;
		if (this.tags.tracktype) {
			grade = this.tags.tracktype.charAt(5);
		}
		if (this.tags.tunnel) {
			shp_tunnels.add(this.geom, {
				id: this.id,
				type: this.tags.highway,
				level: this.tags.layer,
				ref: this.tags.ref,
				grade: grade
			});
		} else if (this.tags.bridge) {
			shp_bridges.add(this.geom, {
				id: this.id,
				type: this.tags.highway,
				level: this.tags.layer,
				ref: this.tags.ref,
				grade: grade
			});
		} else {
			shp_roads.add(this.geom, {
				id: this.id,
				type: this.tags.highway,
				ref: this.tags.ref,
				grade: grade
			});
		} 
	} else if (this.tags.waterway) {
		shp_lines.add(this.geom, {
			id: this.id,
			type: this.tags.waterway
		});	
	} else if (this.tags.aerialway) {
		shp_lines.add(this.geom, {
			id: this.id,
			type: this.tags.aerialway
		});	
	} else if (this.tags.route) {
		shp_lines.add(this.geom, {
			id: this.id,
			type: this.tags.route
		});	
	} 
}
 
Osmium.Callbacks.area = function() {
	if (this.tags.amenity) {
		shp_areapois.add(this.geom, {
			id: this.id,
			type: this.tags.amenity,
			name: this.tags.name
		});
 
	}	
	else if (this.tags.landuse) {
		shp_background.add(this.geom, {
			id: this.id,
			type: this.tags.landuse
		});
	}
	else if (this.tags.natural && this.tags.natural.match(/^(beach|heath|water|wetland|wood)$/)) {
		shp_background.add(this.geom, {
			id: this.id,
			type: this.tags.natural
		});
	}
	else if (this.tags.waterway && this.tags.waterway == 'riverbank') {
		shp_background.add(this.geom, {
			id: this.id,
			type: this.tags.waterway
		});
	}
}
 
Osmium.Callbacks.end = function() {
	shp_pois.close();
	shp_areapois.close();
	shp_places.close();
	shp_tunnels.close();
	shp_bridges.close();
	shp_roads.close();
	shp_lines.close();
	shp_background.close();
}

Since each shapefile dataset can only hold one datatype (point, line or polygon), we need at least three shapefile sets. I decided to further break down linear objects into bridges, tunnels, roads and other linear objects not directly related to the road network like rivers. There is also an additional polygon layer to hold the points of interests (POI) that are areas instead of points.

Multipolygons

Luckily Osmium has the ability to convert multipolygons properly. To achive this, osmjs is executed like this

./osmjs -2 -m  -l sparsetable -j config.js data.osm.pbf

where config.js is the name of your configuration file and data.osm.pbf the osm data in protobuf format.

Relations

I have not yet found out how to process other relations, e.g. hiking routes, in a way that would allow to copy their attributes to the members.

Big, big thanks to User:Joto for the tool as well as for the article in the German IT magazin "Ix" that finally got me going!

Import into QGIS

One important thing is to get the projection right. I found it easiest to add a WMS layer first, e.g. the free osm tiles from http://wms.latlon.org/. Set the projection to EPSG:3857 as listed there, and activate on the fly reprojection.

Then you can import the generated shapefiles into QGIS (not necessarily under linux, QGIS is available for windows and mac as well). If they align well, you may disable or remove the WMS layer.

Add the layers in the order you would like them to be drawn, or move them around later and start to assign styles to the different highways, areas and so on. Note: If you have non-ascii characters e.g. in name fields, make sure the character set is set to UTF-8 when importing.

To define different line styles within one layer, you would open the layer properties and set the style to rule based. After that you can add a filtering rule for each line type you want and set its style indivdually.

In my example, bridges and normal roads are separated because of the drawing order only, the lines styles are supposed to be identical in both layers. So, I defined the styles for the road layer first, saved them to disk and loaded them into the bridges layer then.

Icons for the POI layers are as well easy to import, you only need to drop svg files into the QGIS subdirectory apps->qgis->svg of your installation.

If you want your icons, line width etc. to be resized when zooming (i.e. relative to the map size), you need to choose "Map unit". Note that this works best with a projection that uses metric units instead of degrees (EPSG:3857 does, among others), otherwise a map unit is huge and you often cannot set reasonable small sizes because the number of decimals allowed in the forms is limited.

To add icons to the area POIs change the default "Simple fill" to "Centroid fill", then in the change marker dialog switch from "Simple marker" to "SVG marker" to have all icons available.

I admit that it is a lot of work to set up all the line styles, area fillings and icons, but the good things about it:

• updating the map is easy, as QGIS only references the shape files. Thus, you can generate new ones and simply replace the old files
• making a map from another area should be similarly easy (but I guess every new map will reveal something that is not perfect yet)
• and you can share your style definitions with others (best along with the osmjs configuration file because the data structure must match)

Additional data

When making maps of islands or coastal areas, objects with natural=coastline are not always helpful as creating proper polygons from them may fail. Luckily, the coastline file mapnik uses for higher zoom levels is available from http://tile.openstreetmap.org/processed_p.tar.bz2. To cut the huge coastline shapefile down to the region of interest, select the coastline objects you would like to keep and use "Clip" from QGIS Geoprocessing tools. This works well at least for islands. Openstreetmapdata.com provides sea polygons, land polygons and coastlines as well.

Contour lines can be generated from SRTM or ASTER data. When using the latter, don't forget correct attribution. Gdal tools for conversion should be already installed if Osmium works. You can use gdal_translate to create geotiffs from SRTM hgt files, gdalwarp to combine two or more geotiffs, and finally e.g.

gdal_contour -a height infile.tif outfile.shp -i 20

to create a shapefile with contour lines every 20 m.

Exporting from QGIS

QGIS' print composer lets you define areas to export into svg or pdf files. Version 1.7.4 throws a warning about svg export being buggy, but I encountered some issues with the pdf export as well. One is easy to solve: the background color from the normal view is missing in the print composer, you need to set it separately (tab "item").

Then I had trouble with the labels, the spacing between the letters was completely broken. This does not seem to be related to the font and happened both under windows and linux, rather some weird resizing effect when using very, very small font sizes. My current workaround is to export the labels separately to svg, convert it (after some editing) to pdf with Inkscape and then combine both pdfs to one. Not too convenient as it requires some additional manual steps to execute whenever the map is updated.

Similarly, very thin lines (e.g. 0.5 map units with units set to meter) look fine within QGIS, but they are considerably thicker after exporting.

The labels I use for the contour lines behaved somewhat strange as well: defined in map units, they get smaller when zooming out (just as expected) until a certain zoom level when they suddenly get huge. This happened in the print composer as well until I changed the resolution to 72 dpi. This problem is not reproducable right now though.

Results and QGIS styles

• example map showing El Hierro (western Canary Islands)
• the styles and osmium configuration file