User:SomeoneElse/Ubuntu 1404 tileserver load

From OpenStreetMap Wiki
Jump to navigation Jump to search

After setting up the tileserver as described in http://wiki.openstreetmap.org/wiki/User:SomeoneElse/Ubuntu_1404_tileserver I then loaded data into it.

I also wanted to the resulting map to display England-and-Wales rights of way (which are stored in a "designation" tag). I had already created a lua tag transform script to do this https://github.com/SomeoneElseOSM/designation-style . That works with a standard "openstreetmap-carto" map style https://github.com/gravitystorm/openstreetmap-carto , but changes the contents of the data before loading into the rendering database so that the style that's defined in openstreetmap-carto for "highway=footway" I use for "designation=public_footpath".

Installing and building Osmosis

The virtual machine on which it is installed only has 2Gb of virtual memory, so even a full extract of England would be too large to maintain. I therefore used osmosis to cut an area out of an appropriate Geofabrik extract.

That is available in the repository of 14.04, but as with the main installation, I've built from source here.

Osmosis requires Java. Initially I tried "sudo apt-get install openjdk-7-jre-headless" but eventually found that I needed:

  • sudo apt-get install openjdk-7-jdk


and then did

  • cd ~/src/osmosis
  • ./gradlew assemble
  • sudo ln -s /home/myusername/src/osmosis/package/bin/osmosis /usr/local/bin/

Extracting the data to load

I worked out the northernmost and southernmost latitudes to use and the easternmost and westernmost latitudes by looking at the values in the URL on the osm.org website. Osmosis usage is described here: http://wiki.openstreetmap.org/wiki/Osmosis#Example_Usage . This means that I can do:

  • cd ~/data
  • osmosis --read-pbf ./great-britain-latest.osm.pbf --bounding-box top=54.623 left=-2.31923 bottom=52.35043 right=0.1672 --write-pbf cut.pbf

This gives this warning: 

WARNING: Attention: Data being output lacks metadata. Please use omitmetadata=true

But I won't worry about that - nothing that I'm going to use this data for currently needs metadata (who edited a way last, for example). Eventually I see: 

INFO: Total execution time: 165970 milliseconds.


Get the tag transform script

Next grab the lua tag transform script:


Reload the data and clear previously cached tiles

An osm2pgsql "create" throws away all existing data and replaces it, so don't do this if you actually want to "merge" (and if you do want to merge, be aware of potential problems if ways exist in both your old and new data). You don't need to drop and recreate the rendering database itself to reload it, so I can just do this:

  • osm2pgsql --create --slim -C 1500 --number-processes 1 -S ../openstreetmap-carto/openstreetmap-carto.style --multi-geometry --tag-transform-script /home/myusername/data/style.lua cut.pbf


... and while that's running, remove old tiles from cache. Previously rendered tiles are stored in numbered directories (the first level of which is zoom level) below 

/var/lib/mod_tile/default

(they're owned by whoever you are running the tile server as)

So cd there and delete cached directories below it if you want to clear the cache out. I do want to do this here because I'm changing the map style. If you're not doing that, you probably wouldn't want to clear the cache.


Eventually osm2pgsql will finish, and you'll see this message: 

Osm2pgsql took 2914s overall

If it "appears to finish", and you don't see this message, something probably went wrong.

Next, bounce mapnik and apache:

  • sudo /etc/init.d/renderd restart
* Restarting Mapnik rendering daemon renderd                            [ OK ]
  • sudo /etc/init.d/apache2 restart
* Restarting web server apache2                                                [Sun Apr 27 15:16:33.926531 2014] [tile:notice] [pid 18722:tid 140536259155840] Loading tile config default at /osm_tiles/ for zooms 0 - 20 from tile directory /var/lib/mod_tile with extension .png and mime type image/png
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
                                                                        [ OK ]

Testing

First trying looking at a high-zoom tile that I know is in the area that should be rendered, and should look different from the standard style. In my case this (a location in Derbyshire where there's a "highway=track" with a "designation=public_footpath") will work:

I found the relevant parts of the tile URL by going to the main osm.org website, finding a place that should render differently with my style, zooming in to zoom 18, turning off all layers, right-clicking the map and selecting "view image". That gave me "http://c.tile.openstreetmap.org/18/129790/85317.png", and from there I was able to create the URL for my server.

The reason that I picked zoom 18 is that tiles render much faster at that zoom level than at lower zooms. You can keep an eye on render requests by doing:

  • sudo tail -f /var/log/syslog

Note that in here you may see debug messages such as: 

Jun  2 09:12:53 myserver renderd[17054]: DEBUG: Failed to read cmd on fd 10

Don't worry about those - the issue is described here: https://github.com/openstreetmap/mod_tile/issues/77 .


Similarly I can also do:

and

just as you can with the main OSM website.

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=User:SomeoneElse/Ubuntu_1404_tileserver_load&oldid=1081541"