User:SomeoneElse/Ubuntu 1404 tileserver
I recently installed a small (2Gb memory in a virtual machine) tile server on Ubuntu 14.04. I intend to use it for serving maps of a slightly customised "standard" style covering the middle of England (Oxford to Cleveland), but initially I just installed the Geofabrik extract for "Derbyshire". This essentially is what I did.
I've changed the actual IP address that I used to "myserveraddress" in what follows, my user name to "myusername", and my server name to "myservername".
It's based on the previous "switch2osm" instructions:
http://switch2osm.org/serving-tiles/manually-building-a-tile-server-12-04/
and incorporates a number of updates to that (mostly from pnorman). Thanks to everyone who's contributed directly or indirectly - almost none of what follows is my own work.
The idea was to rely as much as possible on Ubuntu repositories rather than PPAs, and to build the OSM-specific stuff from source (just to reduce external dependancies). The one PPA that was needed was for "nodejs" (see below). If you try and follow these notes on a server with other PPAs in place (such as might happen if you've installed TileMill) things probably won't work exactly as described below, and you might end up with things in different places.
Server installation
I downloaded and installed Ubuntu 14.04 LTS from here: http://www.ubuntu.com/download/server
Of the options available at installation, I selected only "OpenSSH server". I also installed "emacs23-nox" to use as an editor. That brings down quite a lot - if you install a different main editor you may see other dependancies lower down.
After installation, just to make sure that it was up to date I then did:
- sudo -i
(all the apt-get commands need to be run as root)
- apt-get update
- apt-get upgrade
Packaged Software
Next, I installed the minimal set of software required by a "switch2osm-style" server:
- apt-get --no-install-recommends install -y python-software-properties git unzip curl build-essential
I installed Postgres from Ubuntu's repositories:
- apt-get install postgresql-9.3-postgis-2.1 postgresql-contrib-9.3 proj-bin
Ubuntu 14.04 contains Apache 2.4, to install that I did:
- apt-get install apache2 apache2-mpm-worker
- apt-get install munin-node munin munin-plugins-extra libdbd-pg-perl sysstat iotop ptop
- sed -i "s|Allow from.*|Allow from all|" /etc/munin/apache.conf
- service apache2 reload
Next, because I was installing some things from source I needed various development packages. This list is consolidated from what I found that I needed as I went along:
- sudo apt-get install autoconf apache2-dev libtool libxml2-dev libbz2-dev libgeos-dev libgeos++-dev libproj-dev gdal-bin libgdal1-dev mapnik-utils python-mapnik libmapnik-dev
And some things that osm2pgsql will need now (protobuf, to handle .pbf files) or that I'll need later (lua, because at some stage in the future I'll reload data with https://github.com/SomeoneElseOSM/designation-style).
- sudo apt-get install libprotobuf-c0-dev protobuf-c-compiler lua5.2 liblua5.2-dev
Software built from source
The next thing that I did was to get osm2pgsql from source.
- mkdir ~/src
- cd ~/src
- git clone git://github.com/openstreetmap/osm2pgsql.git
- cd osm2pgsql
- ./autogen.sh
- ./configure
- make
- sudo make install
Checking versions
myusername@myservername:~/src/osm2pgsql$
- proj
Rel. 4.8.0, 6 March 2012 usage: proj [ -beEfiIlormsStTvVwW [args] ] [ +opts[=arg] ] [ files ]
- psql --version
psql (PostgreSQL) 9.3.4
- grep 'default_version' '/usr/share/postgresql/9.3/extension/postgis.control'
default_version = '2.1.2'
- osm2pgsql --version
osm2pgsql SVN version 0.85.0 (64bit id space)
Although probably not needed for such as small OSM data extract, I then set up memory overcommit as per previous instructions:
Edit:
- /etc/sysctl.d/60-overcommit.conf
Adding:
# Overcommit settings to allow faster osm2pgsql imports vm.overcommit_memory=1
Then:
- sudo sysctl -p /etc/sysctl.d/60-overcommit.conf
Create a database:
From
myusername@myservername:~/src/osm2pgsql$
I did:
- sudo -u postgres createuser -s $USER
- createdb gis
- psql -d gis -c 'CREATE EXTENSION hstore; CREATE EXTENSION postgis;'
which just outputs:
CREATE EXTENSION
then:
- sudo munin-node-configure --sh | sudo sh
- sudo service munin-node restart
Get the OSM stylesheet:
(I did this from "~", but it might be better done from somewhere like "~/data", where I put the OSM data extract below)
Which returns:
Cloning into 'openstreetmap-carto'... remote: Reusing existing pack: 1735, done. remote: Total 1735 (delta 0), reused 0 (delta 0) Receiving objects: 100% (1735/1735), 712.09 KiB | 411.00 KiB/s, done. Resolving deltas: 100% (860/860), done. Checking connectivity... done.
Fonts needed by the OSM stylesheet:
There's a section of the OSM stylesheet installation guide that talks about the fonts that are needed:
DejaVu Sans and Unifont are the two required fonts. The others are only needed if you actually expect to need to display those fonts - if you don't, you'll just see something like "square characters" in names that need those fonts to display correctly.
Loading an OSM data extract.
Initially I just used a smaller county extract from Geofabrik:
- mkdir data
- cd data
- wget http://download.geofabrik.de/europe/great-britain/england/derbyshire-latest.osm.pbf
I loaded the data (the path reference to openstreetmap-carto.style may need to be corrected to the actual location of the "openstreetmap-carto.style" file):
- osm2pgsql --create --slim -C 1500 --number-processes 1 -S ../openstreetmap-carto/openstreetmap-carto.style --hstore --multi-geometry ./derbyshire-latest.osm.pbf
Things that need nodejs
(I did this section from the "myusername@myservername:~/data$ " directory)
I initially tried to use the Ubuntu-packaged nodejs, but that didn't work, so I then did:
- sudo add-apt-repository -y ppa:chris-lea/node.js
- sudo apt-get update
- sudo apt-get upgrade
- sudo apt-get install nodejs
- npm install npm
- npm install millstone carto
That got me:
- ./node_modules/carto/bin/carto --version
carto 0.9.6 (Carto map stylesheet compiler)
- cd ~/openstreetmap-carto
(which would have been ~/data/openstreetmap-carto if I'd put it where I intended to)
- ./get-shapefiles.sh
'"renderd" and "mod_tile"'
The "autogen" below may have problems if it's run on a slightly different system, or if the packages that make up Ubuntu have changed between me writing and you reading this, or if mod_tile has changed (all of which are possible). If something goes wrong, drop me a message, or try the usual help channels (#osm and #osm-dev on IRC, the OSM help site, etc.).
- cd ~/src
- git clone git://github.com/openstreetmap/mod_tile.git
- cd mod_tile
- ./autogen.sh
- ./configure
- make
- sudo make install
- sudo make install-mod_tile
- sudo ldconfig
Next, I edited
/usr/local/etc/renderd.conf
and changed
plugins_dir=/usr/lib/mapnik/input
to
plugins_dir=/usr/lib/mapnik/2.2/input
I installed the command-line "TileMill to Mapnik stylesheet converter" carto:
- sudo npm install -g carto
and checked that it ran OK:
- type carto
carto is /usr/bin/carto
I changed directory to the location of project.mml (which in my case was):
- cd ~/openstreetmap-carto/
and ran
- carto project.mml > mapnik.xml
Next I edited renderd.conf so that it contains the path to this:
- XML=/home/myusername/openstreetmap-carto/mapnik.xml
(and also I set HOST=localhost in there)
I manually created the directory for generated tiles:
- sudo mkdir /var/lib/mod_tile
- sudo chown myusername /var/lib/mod_tile
and also the directory for the socket used to communicate from the apache module (that realises that it needs a tile) to renderd (that actually renders it):
- sudo mkdir /var/run/renderd
- sudo chown myusername /var/run/renderd
This needs creating on every reboot, but is looked after by the "init" script once that is set up. See below for that. It was done manually here because I initially ran renderd in debug mode from the shell.
Apache setup
In Apache 2.4, "conf-available" and "conf-enabled" have replaced "conf.d".
I created
- /etc/apache2/conf-available/mod_tile.conf
containing
LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so
and then did:
- sudo a2enconf mod_tile
Enabling conf mod_tile. To activate the new configuration, you need to run: service apache2 reload
which created a symlink from "conf-enabled". The "disable" script to match "a2enconf" is "a2disconf".
Note that renderd won't actually run yet, because the config info it needs hasn't been set up yet.
In "/etc/apache2/sites-available/000-default.conf" after the server email address line I added:
- LoadTileConfigFile /usr/local/etc/renderd.conf
- ModTileRenderdSocketName /var/run/renderd/renderd.sock
- # Timeout before giving up for a tile to be rendered
- ModTileRequestTimeout 0
- # Timeout before giving up for a tile to be rendered that is otherwise missing
- ModTileMissingRequestTimeout 30
and ran:
- sudo service apache2 reload
This isn't ideal, because if you run "a2disconf mod_tile" a "service apache2 reload" will have problems because the "000-default.conf" has errors in it, because "LoadTileConfigFile" is defined by the disabled "mod_tile" config. The error is something like this:
- sudo a2disconf mod_tile
Conf mod_tile disabled. To activate the new configuration, you need to run:
service apache2 reload
- sudo service apache2 reload
Reloading web server apache2 * The apache2 configtest failed. Not doing anything. Output of config test was: AH00526: Syntax error on line 13 of /etc/apache2/sites-enabled/000-default.conf: Invalid command 'LoadTileConfigFile', perhaps misspelled or defined by a module not included in the server configuration Action 'configtest' failed. The Apache error log may have more information.
I'm sure that there's a proper Apache 2.4 way of fixing this; I haven't found it yet, and as I have no intention of running the web server without mod_tile, it's not a requirement for me.
At this point I had a look at the end of /var/log/apache2/error.log to make sure that nothing untoward was happening.
[Sat Apr 26 12:31:37.626116 2014] [mpm_event:notice] [pid 1182:tid 140453313202048] AH00493: SIGUSR1 received. Doing graceful restart [Sat Apr 26 12:31:37.713271 2014] [tile:notice] [pid 1182:tid 140453313202048] Loading tile config default at /osm_tiles/ for zooms 0 - 20 from tile directory /var/lib/mod_tile with extension .png and mime type image/png AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message [Sat Apr 26 12:31:37.727167 2014] [mpm_event:notice] [pid 1182:tid 140453313202048] AH00489: Apache/2.4.7 (Ubuntu) configured -- resuming normal operations [Sat Apr 26 12:31:37.728041 2014] [core:notice] [pid 1182:tid 140453313202048] AH00094: Command line: '/usr/sbin/apache2'
Test renderd
I then ran renderd interactively from the command line:
- renderd -f -c /usr/local/etc/renderd.conf
(no obvious errors occurred)
And elsewhere I browsed to
and a tile was indeed visible.
Running renderd in the background
Next I set up renderd to run in the background from init:
- sudo cp ~/src/mod_tile/debian/renderd.init /etc/init.d/renderd
- sudo chmod u+x /etc/init.d/renderd
I edited renderd.init to contain:
- DAEMON=/usr/local/bin/$NAME
- DAEMON_ARGS="-c /usr/local/etc/renderd.conf"
and in that file I also changed "www-data" to "myusername" above.
When I ran:
- sudo /etc/init.d/renderd start
* Starting Mapnik rendering daemon renderd iniparser: syntax error in /usr/local/etc/renderd.conf (7):
-> ;[renderd01]
iniparser: syntax error in /usr/local/etc/renderd.conf (14):
-> ;[renderd02]
iniparser: syntax error in /usr/local/etc/renderd.conf (33):
-> ;** config options used by mod_tile, but not renderd **
iniparser: syntax error in /usr/local/etc/renderd.conf (44):
-> ;[style2]
iniparser: syntax error in /usr/local/etc/renderd.conf (51):
-> ;** config options used by mod_tile, but not renderd **
[ OK ]
(apart from objecting to semicolons) it starts OK.
I then rebooted to check that everything restarted OK - and it did.