Ubuntu tile server
This page describes a set of Ubuntu packages that help making installing one's own OSM tile server easier
Contents |
Introduction
This is an early version, so there might still be some problems with it. These packages work with either Ubuntu 11.04 (Natty Narwhal) or Ubuntu 11.10 (Oneiric Ocelot).
After installation, you should have your own working tileserver with the standard OSM-mapnik stylesheet into which you can import an extract of the OSM data for rendering.
It is based on the same software that the main OSM tileserver uses. I.e. mod_tile for serving renderd as the rendering daemon, mapnik for the actual rendering. The main aim of these packages is to simplify installation, by automating as much as possible into packaging scripts.
Installation
The following commands need to be entered into the terminal to set things up:
Add the repository containing the packages:
sudo add-apt-repository ppa:kakrueger/openstreetmap
(If you don't have add-apt-repository, sudo apt-get install python-software-properties)
Update the local package list to pick up the new repository:
sudo apt-get update
Install the package libapache2-mod-tile and its dependencies. During configuration, it will ask you few questions. To make sure the automatic setup scripts work, you should keep the defaults. However, in the question about permissions for users of the database, you want to add your own username after the user www-data separated by a space to be able import data under your user.
sudo apt-get install libapache2-mod-tile
Planet Import
Download the OpenStreetMap data extract you want to render. e.g:
wget http://download.geofabrik.de/osm/europe/ireland.osm.pbf
Import the data into a postgresql database. There are a number of different parameters you can use with osm2pgsql that depend on your available hardware and the size of the data extract you want to import. -C specifies the number of Mb osm2pgsql will use to cache data. So this depends on the amount of RAM you have available.
osm2pgsql --slim -C 1500 ireland.osm.pbf
Important: depending on the size of the extract you are importing and the performance of the computer this can take up to multiple days! If you are importing the full planet, you must set -C 12000 (may increase as the OSM database grows), this will cause your server to swap during the import if you don't have a substantial amount of memory (16+GB), however this is still an order of magnitude faster than using smaller values for the node cache. For an initial planet import it also may make sense to temporarily change your PostgreSQL configuration; for example increase the number of checkpoint segments, reduce shared buffer size, and possibly turn off fsync, autovacuum, and synchronous_commit.
Afterwards you need to restart the rendering daemon.
sudo /etc/init.d/renderd restart
If everything worked OK, you should have a working tileserver and you can view its results by going to http://localhost/osm/slippymap.html
Updating
The following commands explain how to keep your tile server up-to-date with the latest OSM data.
After importing the initial database with osm2pgsql as described above (you will need to have used the --slim option in the initial import to allow for updating), you need to do the following steps
Install osmosis
sudo apt-get install osmosis
Grant rights to the update tables to user www-data. Note that this must be done after the planet import is complete or subsequent updates may fail due to missing permissions on late-created relations.
sudo /usr/bin/install-postgis-osm-user.sh gis www-data
Initialize the osmosis replication stack to the data of your data import (full planet dumps take more than 24 hours to generate, as a consequence you should start the updates beginning one day before the date of the planet file).
sudo -u www-data /usr/bin/openstreetmap-tiles-update-expire.sh 2011-10-25
Update your tileserver by up to an hour and expire the corresponding rendered tiles
sudo -u www-data /usr/bin/openstreetmap-tiles-update-expire.sh
Note: As of 2012/01/23, /usr/bin/openstreetmap-tiles-update-expire.sh has an invalid value for the -s switch on render_expired. It is missing the renderd subdirectory. It should be:
-s /var/run/renderd/renderd.sock
Failure to repair this will result in the dirty tiles NOT actually being processed and a subsequent execution of ...update-expire will overwrite the expired tile list.
If your tile server is behind more than an hour you will need to call the openstreetmap-tiles-expire script multiple times. If you want to continuously keep your server up to data, you need to add the openstreetmap-tiles-expire.sh script to your crontab.
Note: keeping the data up-to-date is the major challenge, in particular because after the import you may already be multiple days behind. Consider changing maxInterval in /var/lib/mod_tile/.osmosis/configuration.txt to 21600 (six hours) till you have caught up, further add "--number-processes 2" to the osm2pgsql command in /usr/bin/openstreetmap-tiles-update-expire.sh or a higher number if this is appropriate for your hardware.
Troubleshooting
There are a number of things that can go wrong. Here are a few of the common problems and their solutions:
Connection to database failed in osm2pgsql
If you are getting the following error message when importing data with osm2pgsql, you have probably forgotten to add your username in the allowed users section during the configuration. Connection to database Failed: FATAL: Ident authentication failed for user "xyz"
You can fix this in one of two ways:
Either you reconfigure the package, this time including both www-data and your own user name in a space separated list,
sudo dpkg-reconfigure openstreetmap-postgis-db-setup
Or, you manually grant permissions to your user account with the following command:
sudo /usr/bin/install-postgis-osm-user.sh gis xyz
Hardware
Hardware requirements can be quite demanding if you want to render larger areas, but aren't too bad if you are only interested in smaller regions. For a standard desktop (approximately 4 GB of ram, standard harddisk, dual - quad core CPU) probably an extract size of about 100 - 300 Mb is reasonable (import time of the order of an hour).
If you want to import and render the whole world, you will need a considerably beefier server than a typical desktop. E.g. starting from about 24Gb of RAM upwards and a raid array of harddisks or preferably an SSD.
FAQ
Where are the various files (DB / Tiles, etc.) ?
- Style sheets and coastlines are in /etc/mapnik-osm-data
- Tiles are in /var/lib/mod_tile
- Renderd configuration in /etc/renderd.conf
- Renderd socket is /var/run/renderd/renderd.sock
- mod_tile configuration in /etc/apache2/sites-available/tileserver_site
- Scripts are in /usr/bin
- DB configuration is in /etc/postgresql/X.X/main
- (X.X stands for the PostgreSQL version for example 9.1)
- osm2pgsql configuration and state.txt is in /var/lib/mod_tile/.osmosis
How do I pre-render tiles ?
You can use render_list to pre-render tiles:
Usage: render_list [OPTION] ... -a, --all render all tiles in given zoom level range instead of reading from STDIN -f, --force render tiles even if they seem current -m, --map=MAP render tiles in this map (defaults to 'default') -l, --max-load=LOAD sleep if load is this high (defaults to 5) -s, --socket=SOCKET unix domain socket name for contacting renderd -n, --num-threads=N the number of parallel request threads (default 1) -t, --tile-dir tile cache directory (defaults to '/var/lib/mod_tile') -z, --min-zoom=ZOOM filter input to only render tiles greater or equal to this zoom level (default is 0) -Z, --max-zoom=ZOOM filter input to only render tiles less than or equal to this zoom level (default is 18) If you are using --all, you can restrict the tile range by adding these options: -x, --min-x=X minimum X tile coordinate -X, --max-x=X maximum X tile coordinate -y, --min-y=Y minimum Y tile coordinate -Y, --max-y=Y maximum Y tile coordinate Without --all, send a list of tiles to be rendered from STDIN in the format: X Y Z e.g. 0 0 1 0 1 1 1 0 1 1 1 1 The above would cause all 4 tiles at zoom 1 to be rendered
Note that you have to set --socket=/var/run/renderd/renderd.sock as in:
render_list -s /var/run/renderd/renderd.sock -m default -a -z 0 -Z 10
Links
- Switch2OSM - Make the switch to OpenStreetMap Has step by step instructions on how to set up your own tile server (based upon Ubuntu Linux 10.04 LTS). See the section "Serving Tiles".
