Deploying your own Slippy Map Gentoo
On Gentoo things work a little different than on most other Linux distributions. To deploy a slippy map on Gentoo you can follow the steps below, however it is by no means the best or most robust way.
The solution is based on Mapnik, Mod_tile, and Openlayers.
This page is based on Deploying your own Slippy Map.
Overview
There are four main components we need to set up to deploy a slippy map.
- First we need to get the Openstreetmap data and imported it into a postgres database with postgis extension.
- Solutions that try to avoid this step are horrendously slow and only intended for very small areas.
- Osm2pgsql will be used to import the data.
- The second component is the rendering toolchain based on mapnik.
- Thirdly we need to set up a stylesheet,
- contains the reference to the database.
- contains information on how to render the different map elements
- can reference shape files. Which contain additional map data. For example coastlines and other high level features.
- The previous steps create a tile server. Using leaflet we create a web page that renders the slippy map.
Step one: Importing Openstreetmap data
Get the planet file
Get a planet file or get an extract (saves time to download). You can chose the protocol buffer (.osm.pbf) format. These are smaller to download and osm2pgsql can handle them.
Installing database packages
- dev-db/postgresql (13.1)
- dev-db/postgis (3.0.3)
- sci-geosciences/osm2pgsql (1.3.0)
Previously the pbf useflag was required on osm2pgsql however this is now supported by default
Postgresql is slotted, but if older versions are not needed for other projects, it is easiest to only install one version.
In make.conf add POSTGRES_TARGETS
POSTGRES_TARGETS="postgres13"
Install the database packages
emerge dev-db/postgresql dev-db/postgis sci-geosciences/osm2pgsql
Creating database and user
As the database is pretty large you might not want to store it in the default location (own partition maybe).
First su into the postgres user:
su - postgres
Create the directory where you want the database and initialize it
mkdir /path/to/new/dir initdb13 --pgdata /path/to/new/dir
it may be needed to copy the config files into /etc/postgres13
mv /path/to/new/dir/*.conf /etc/postgresql-13
touch /path/to/new/dir/postgresql.auto.conf
Edit /etc/conf.d/postgresql-13 to make DATA_DIR point to the /path/to/new/dir and start the postgres server.
Create a new user and a new database:
createuser <username>
createdb -E UTF8 -T template0 -O <username> <db>
Install the postgis extension for the database
psql13 <db>
CREATE EXTENSION postgis;
ALTER TABLE geometry_columns OWNER TO <username>;
ALTER TABLE spatial_ref_sys OWNER TO <username>;
\q
Permissions
Make sure you add the users that are going to use the database (user running osm2pgsql, the webserver) to the postgres group
gpasswd -a <username> postgres
Importing using osm2pgsql
Now that the database is all set lets fill it
osm2pgsql -U <user> -m -d <db> <planet-file>
If you only want a section of the world do:
osm2pgsql -U <user> --bbox minlon,minlat,maxlon,maxlat -m -d <db> <planet-file>
Step two: Rendering toolchain
Use Flags
Set the following useflags:
- geos
- gdal
- proj
- postgres
- python
Required packages
The following packages from portage will be used Version numbers are included to document a working combination. Newer should also work.
- dev-vcs/git
- www-servers/apache (2.4.23)
We will also need Mapnik and mod_tile some of which used to be in portage, but none are currently.
- sci-geosciences/mapnik (3.0.9)
- www-apache/mod_tile (9999 live ebuild)
ebuild and overlay contains a mod_tile ebuild. You can add the overlays. Or fetch it directly fromfrom here and add it to a local overlay.
Currently the live ebuild for mod_tile is broken. The examples have been moved. Since we don't need them we can remove these lines from the ebuild.
dodoc readme.txt
doman docs/*.1
insinto /var/www/osm
newins slippymap.html index.html
Currently the ebuild for mapnik depends on agg. Portage only contains agg-2.5 or higher, while only up until agg-2.4 is it released under a more permissive license. Mapnik bundles this older version of the library and as such this DEPENDS can be dropped.
emerge dev-vcs/git sci-geosciences/mapnik www-servers/apache www-apache/mod_tile
configure apache and mod_tile
In /etc/conf.d/apache2 add -D MOD_TILE to the APACHE2_OPTS. example:
APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D MOD_TILE"
edit the file /etc/apache/modules.d/14_mod_tile.conf and update the location of renderd.sock
ModTileRenderdSocketName /var/run/renderd/renderd.sock
Configure renderd
Mod_tile consists of 2 components, the apache module, and a rendereing daemon renderd. edit /etc/renderd.conf in particular set
[renderd]
stats_file=/run/renderd/renderd.stats
socketname=/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/cache/renderd/tiles
[mapnik]
plugins_dir=/usr/lib64/mapnik/input
font_dir=/usr/share/fonts/
font_dir_recurse=true
[default]
URI=/osm/
XML=/path/to/mapnik-stylesheets/osm.xml
Ensure the referenced paths exist
mkdir /run/renderd
mkdir -p /var/cache/renderd/tiles
the XML file will be created in the next step.
Mapnik python bindings
This step is optional, it allows for easy testing of mapnik rendering, without using mod_tile/apache. It can be skipped.
Starting with mapnik 3 python bindings are no longer included in the mapik library. We will install python-3.4 bindings.
Set up a virtual python environment
python3 -mvenv venv
source env/bin/activate
Your prompt should be prefixed with '(env)'.
clone and build python-mapnik
git clone https://github.com/mapnik/python-mapnik
git checkout v3.0.x
BOOST_PYTHON_LIB=boost_python39 ./setup.py develop build
BOOST_PYTHON_LIB=boost_python39 ./setup.py develop install
To test if the python bindings work
python
>>> import mapnik
>>>
Should complete with no errors.
Step Three: Map style
We will use the old openstreetmap-mapnik style in this article. For simplicity as it avoids the need for node.js. OSM-bright and openstreetmap-carto are some newer alternatives.
Get the style
git clone https://github.com/openstreetmap/mapnik-stylesheets
Get the world boundries
Besides the .osm.pbf stylesheets often also rely on shape files. Which contain information on coastlines, and other features. Mapnik-stylesheets contains a script get-coastlines.sh which fetches the required files. Some are no longer available from the original source. Currently some can be found [[1]]
Fetch and extract all the archives into /usr/share/mapnik/world_boundaries/ or any folder. Different styles may use additional shape files(.shp). In case any of them come without an index(.index) file you can use the shapeindex tool (part of mapnik) to generate them.
Generate osm.xml
The stylesheet that mapnik uses osm.xml includes several files from the inc/ subdirectory. These include files need to be configured to know where to find the postgresql database and shape files.
Since this is the old style format it is not python3, and we'll use python2 to save the trouble of having to do it manually
python2 generate_xml.py --host localhost --password "" --port 5432 --user <username> --dbname <db> --world_boundaries=/usr/share/mapnik/world_boundaries/
If it is not possible to use python2, using 2to3 works well, and only requires minor fixes. adding .encode("utf8")/.decode("utf8") around file access
Generating the first tile
This step is optional and requires mapnik-python to be installed.
Make sure postgres is still running. Since we installed python bindings for python3 we will make mapnik-stylesheets/livetile/render_single_tile.py python3 compatible by changing:
sys.stdout.write(im.tostring('png'));
to
open('firsttile.png', 'wb').write(im.tostring('png'))
you should now be able to render your first tile with the command:
python render_single_tile.py ../../mapnik-stylesheets/osm.xml 0 0 0
Slippy map
start apache2 and check that mod_tile is loaded by visiting http://localhost/mod_tile.
you can start renderd via its init script. but to see what's going on start it from shell with the -f flag
/etc/init.d/apache2 start
renderd -f
Pointing your browser to http://localhost/osm/0/0/0.png should show a tile.
Step Four: Slippy map in browser
Download leaflet and place it in a leaflet/ subfolder in the apache document root.
Add a html file with a slightly modified version of the mod_tile example.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="leaflet/leaflet.css" />
<script src="leaflet/leaflet.js"></script>
</head>
<body style="margin: 0;">
<div id="map" style="position: absolute; width: 100%; height: 100%;"></div>
<script>
var map = L.map('map').setView([0, 0], 4);
L.tileLayer('/osm/{z}/{x}/{y}.png', {
maxZoom: 18,
attribution: 'Map data © <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors.',
}).addTo(map);
</script>
</body>
</html>