OpenHistoricalMap/Dev

From OpenStreetMap Wiki
Jump to navigation Jump to search
Logo.png
This page describes a historic artifact in the history of OpenStreetMap. It does not reflect the current situation, but instead documents the historical concepts, issues, or ideas.
About
OpenHistoricalMap's original server-side configuration
Reason for being historic
The server-side configuration is now tracked in various GitHub readmes. See OpenHistoricalMap/Development for starting points.
Captured time
2015


Introduction

Documentation of things for OHM development


OHM Github repo: https://github.com/organizations/OpenHistoricalMap

Key pieces of infrastructure: - Rails Port - Mod_tile


Server Setup

The server was mostly set up with packages according toː http://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/

Originally the older mapnik osm.xml stylesheet in /home/tim/src/mapnik-style/osm.xml was used

Then the ohm-carto carto css was deployed. This currently lives in /home/tim/ohm-carto/

The shapefiles I downloaded and put into ohm-carto/data.zip which need to be extracted into ohm-carto/data to use.

the carto node module was installed into this ohm-carto directory.

The mapnik.xml was created using the carto command:

:~/ohm-carto$ ./node_modules/carto/bin/carto project.mml > mapnik.xml

The default tiles site was disabled from Apache. Instead we just added the one line to the OHM VHOST:

LoadTileConfigFile /etc/renderd.conf

The renderd conf:

[renderd]
stats_file=/var/run/renderd/renderd.stats
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile

[mapnik]
plugins_dir=/usr/lib/mapnik/input
font_dir=/usr/share/fonts/truetype/
font_dir_recurse=true

[default] 
XML=/home/tim/ohm-carto/mapnik.xml
#XML=/home/tim/src/mapnik-style/osm.xml
URI=/ohm_tiles/
DESCRIPTION=This is the standard osm mapnik style
HOST=openhistoricalmap.org


Replication / Planet

Osmosis was installed outside the packages and is installed in /usr/local/bin

OHM API -> Replication change files: home/tim/replication/minute/data

every 5 minutes

crontab for tim user

 */5 * * * * cd /home/tim/replication/minute/; osmosis -q --replicate-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-replication workingDirectory=data

Replication change files -> mapnik database and refresh tiles etc:

crontab for www-data user

*/5 * * * * /var/lib/mod_tile/openstreetmap-tiles-update-expire.sh
 /var/lib/mod_tile:

 configuration.txt - points to baseUrl=file://localhost/home/tim/replication/minute/data
 openstreetmap-tiles-update-expire.sh  - collation, import of changes and expiry of tiles
 replag - a script to show the lag between imports, called by the expire.sh script



Planet can be generated via

osmosis --read-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-xml file="ohm_planet_20150303.osm.bz2" 

as of 3 March 2015 it took around 2 1/2 hrs 7943383 ms

osm2pgsql custom things

Osm2pgsql must be compiled with lua support.

in the https://github.com/openhistoricalmap/osm2pgsql repository - there are two files of interest:

ohm_style.style - here we specify that start_date and end_date should be dates in the database, and to add an extra start_date_year and an end_date_year coluumn

ohm_tag_transform.lua - this is a Lua script to parse out and ensure that the dates are in YYYY-MM-DD formatting. It also parses out the year into the _year column value

example execution:

osm2pgsql --slim -S  ohm_style.style --tag-transform-script ohm_tags_transform2.lua -d ohm_gis -C 16000 --number-processes 4 ohm_test.osm 

Custom layers

OHM Readme

The 2008 and 2009 styles are meant to be used with the OHM osm2pgsql style and tag transforms which use start_date and end_date which are postgresql date types and start_date_year and end_date_year which are number types


import the sql commands to create the two 2008 and 2009 views and add them to geometry columns

psql -d ohm_gis_hack -f views_2008_2009.sql

Compile the carto stylesheets

1. install carto locally

npm install carto

2. compile new project files for each:

./node_modules/carto/bin/carto project_2008.mml > mapnik_2008.xml
./node_modules/carto/bin/carto project_2009.mml > mapnik_2009.xml


3. set up renderd configuation for each

[ohm_2008]
XML=/home/tim/ohm-carto/ohm-carto/mapnik_2008.xml
URI=/ohm_2008/
DESCRIPTION=A custom ohm map showing those things in 2008
HOST=openhistoricalmap.org


[ohm_2009]
XML=/home/tim/ohm-carto/ohm-carto/mapnik_2009.xml
URI=/ohm_2009/
DESCRIPTION=A custom ohm map showing those things in 2009
HOST=openhistoricalmap.org


troubleshooting

Renderd

Run renderd from the command line

sudo service renderd stop 
renderd -f -c /etc/renderd.conf

see what errors you find