Deploying your own Slippy Map Gentoo

From OpenStreetMap Wiki
Jump to navigation Jump to search

On Gentoo things work a little different than on most other Linux distributions. To deploy a slippy map on Gentoo you can follow the steps below, however it is by no means the best or most robust way.

The solution is based on Mapnik, Mod_tile, and Openlayers.

This page is based on Deploying your own Slippy Map.

Overview

There are four main components we need to set up to deploy a slippy map.

  1. First we need to get the Openstreetmap data and imported it into a postgres database with postgis extension.
    • Solutions that try to avoid this step are horrendously slow and only intended for very small areas.
    • Osm2pgsql will be used to import the data.
  2. The second component is the rendering toolchain based on mapnik.
  3. Thirdly we need to set up a stylesheet,
    • contains the reference to the database.
    • contains information on how to render the different map elements
    • can reference shape files. Which contain additional map data. For example coastlines and other high level features.
  4. The previous steps create a tile server. Using leaflet we create a web page that renders the slippy map.

Step one: Importing Openstreetmap data

Get the planet file

Get a planet file or get an extract (saves time to download). You can chose the protocol buffer (.osm.pbf) format. These are smaller to download and osm2pgsql can handle them.

Installing database packages

  • dev-db/postgresql (13.1)
  • dev-db/postgis (3.0.3)
  • sci-geosciences/osm2pgsql (1.3.0)

Previously the pbf useflag was required on osm2pgsql however this is now supported by default

Postgresql is slotted, but if older versions are not needed for other projects, it is easiest to only install one version.

In make.conf add POSTGRES_TARGETS 

POSTGRES_TARGETS="postgres13"

Install the database packages 

emerge dev-db/postgresql dev-db/postgis sci-geosciences/osm2pgsql

Creating database and user

As the database is pretty large you might not want to store it in the default location (own partition maybe).

First su into the postgres user: 

su - postgres

Create the directory where you want the database and initialize it 

mkdir /path/to/new/dir
initdb13 --pgdata /path/to/new/dir

it may be needed to copy the config files into /etc/postgres13 

 mv /path/to/new/dir/*.conf /etc/postgresql-13
 touch /path/to/new/dir/postgresql.auto.conf

Edit /etc/conf.d/postgresql-13 to make DATA_DIR point to the /path/to/new/dir and start the postgres server.

Create a new user and a new database: 

 createuser <username>
 createdb -E UTF8 -T template0 -O <username> <db>

Install the postgis extension for the database 

 psql13 <db>
 CREATE EXTENSION postgis;
 ALTER TABLE geometry_columns OWNER TO <username>;
 ALTER TABLE spatial_ref_sys OWNER TO <username>;
 \q

Permissions

Make sure you add the users that are going to use the database (user running osm2pgsql, the webserver) to the postgres group 

gpasswd -a <username> postgres

Importing using osm2pgsql

Now that the database is all set lets fill it 

osm2pgsql -U <user> -m -d <db> <planet-file>

If you only want a section of the world do: 

osm2pgsql -U <user> --bbox minlon,minlat,maxlon,maxlat -m -d <db> <planet-file>

Step two: Rendering toolchain

Use Flags

Set the following useflags:

  • geos
  • gdal
  • proj
  • postgres
  • python

Required packages

The following packages from portage will be used Version numbers are included to document a working combination. Newer should also work.

  • dev-vcs/git
  • www-servers/apache (2.4.23)

We will also need Mapnik and mod_tile some of which used to be in portage, but none are currently.

  • sci-geosciences/mapnik (3.0.9)
  • www-apache/mod_tile (9999 live ebuild)

ebuild and overlay contains a mod_tile ebuild. You can add the overlays. Or fetch it directly fromfrom here and add it to a local overlay.


Currently the live ebuild for mod_tile is broken. The examples have been moved. Since we don't need them we can remove these lines from the ebuild. 

 dodoc readme.txt
 doman docs/*.1
 insinto /var/www/osm
 newins slippymap.html index.html

Currently the ebuild for mapnik depends on agg. Portage only contains agg-2.5 or higher, while only up until agg-2.4 is it released under a more permissive license. Mapnik bundles this older version of the library and as such this DEPENDS can be dropped. 

 emerge dev-vcs/git sci-geosciences/mapnik www-servers/apache www-apache/mod_tile

configure apache and mod_tile

In /etc/conf.d/apache2 add -D MOD_TILE to the APACHE2_OPTS. example: 

APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D MOD_TILE"


edit the file /etc/apache/modules.d/14_mod_tile.conf and update the location of renderd.sock 

ModTileRenderdSocketName /var/run/renderd/renderd.sock

Configure renderd

Mod_tile consists of 2 components, the apache module, and a rendereing daemon renderd. edit /etc/renderd.conf in particular set 

[renderd]
stats_file=/run/renderd/renderd.stats
socketname=/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/cache/renderd/tiles
[mapnik]
plugins_dir=/usr/lib64/mapnik/input
font_dir=/usr/share/fonts/
font_dir_recurse=true
[default]
URI=/osm/
XML=/path/to/mapnik-stylesheets/osm.xml

Ensure the referenced paths exist 

 mkdir /run/renderd
 mkdir -p /var/cache/renderd/tiles

the XML file will be created in the next step.

Mapnik python bindings

This step is optional, it allows for easy testing of mapnik rendering, without using mod_tile/apache. It can be skipped.

Starting with mapnik 3 python bindings are no longer included in the mapik library. We will install python-3.4 bindings.

Set up a virtual python environment 

 python3 -mvenv venv
 source env/bin/activate

Your prompt should be prefixed with '(env)'.

clone and build python-mapnik 

 git clone https://github.com/mapnik/python-mapnik
 git checkout v3.0.x
 BOOST_PYTHON_LIB=boost_python39 ./setup.py develop build
 BOOST_PYTHON_LIB=boost_python39 ./setup.py develop install

To test if the python bindings work 

 python
 >>> import mapnik
 >>>

Should complete with no errors.

Step Three: Map style

We will use the old openstreetmap-mapnik style in this article. For simplicity as it avoids the need for node.js. OSM-bright and openstreetmap-carto are some newer alternatives.

Get the style

 git clone https://github.com/openstreetmap/mapnik-stylesheets

Get the world boundries

Besides the .osm.pbf stylesheets often also rely on shape files. Which contain information on coastlines, and other features. Mapnik-stylesheets contains a script get-coastlines.sh which fetches the required files. Some are no longer available from the original source. Currently some can be found [[1]]

Fetch and extract all the archives into /usr/share/mapnik/world_boundaries/ or any folder. Different styles may use additional shape files(.shp). In case any of them come without an index(.index) file you can use the shapeindex tool (part of mapnik) to generate them.

Generate osm.xml

The stylesheet that mapnik uses osm.xml includes several files from the inc/ subdirectory. These include files need to be configured to know where to find the postgresql database and shape files.

Since this is the old style format it is not python3, and we'll use python2 to save the trouble of having to do it manually 

python2 generate_xml.py --host localhost --password "" --port 5432 --user <username> --dbname <db> --world_boundaries=/usr/share/mapnik/world_boundaries/

If it is not possible to use python2, using 2to3 works well, and only requires minor fixes. adding .encode("utf8")/.decode("utf8") around file access

Generating the first tile

This step is optional and requires mapnik-python to be installed.

Make sure postgres is still running. Since we installed python bindings for python3 we will make mapnik-stylesheets/livetile/render_single_tile.py python3 compatible by changing: 

sys.stdout.write(im.tostring('png'));

to 

open('firsttile.png', 'wb').write(im.tostring('png'))

you should now be able to render your first tile with the command: 

python render_single_tile.py ../../mapnik-stylesheets/osm.xml 0 0 0

Slippy map

start apache2 and check that mod_tile is loaded by visiting http://localhost/mod_tile.

you can start renderd via its init script. but to see what's going on start it from shell with the -f flag

/etc/init.d/apache2 start
renderd -f

Pointing your browser to http://localhost/osm/0/0/0.png should show a tile.

Step Four: Slippy map in browser

Download leaflet and place it in a leaflet/ subfolder in the apache document root.

Add a html file with a slightly modified version of the mod_tile example.

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="leaflet/leaflet.css" />
    <script src="leaflet/leaflet.js"></script>
</head>
<body style="margin: 0;">
    <div id="map" style="position: absolute; width: 100%; height: 100%;"></div>
    <script>
        var map = L.map('map').setView([0, 0], 4);
        L.tileLayer('/osm/{z}/{x}/{y}.png', {
            maxZoom: 18,
            attribution: 'Map data &copy; <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors.',
        }).addTo(map);
    </script>
</body>
</html>
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Deploying_your_own_Slippy_Map_Gentoo&oldid=2162067"