Read Only Map API
- Not to be confused with Rome
|This article or section may contain out-of-date information. The information is no longer correct, or no longer has relevance.
If you know about the current state of affairs, please help keep everyone informed by updating this information. (Discuss)
"Read only map API" or short ROMA is a subset of the main API only supporting the "map" call, read - only return all objects for a given bounding box. This API is mostly used by the Tiles@home client which needs to fetch the z12 bbox as raw data.
ROMA only defines the usage/API but not the specific implementations. There are multiple implementations available or in development.
Accessing ROMA servers
The ROMA servers can be currently accessed using a load balancer at
Note: This is actually a Trapi-Server and not a Roma-Server.
Resource usage as of 2008-10-23:
- 150GB Disk space
- Minimum 4GB Ram
The Database will heavily profit from more memory as the index on the node table itself is ~11GB so every request even for a single node will most likely hit the disk in this setup. The disks are 4x72GB SCA 10K in a Hardware Stripe - the more spindles the more requests the database will be able to handle at once.
- if you have the cash, you could get an hyperdrive5 to store the index (http://www.hyperossystems.co.uk/)
The Database schema used is the Osmosis 0.5 api simple postgres schema with one slight modification. Osmosis tries to build a bbox for the way after importing. This does never really finish as the tables have not been ANALYZED by postgres and thus it does a full table scan for every single way. As the code itself does not make use of the bbox i simple dropped the bbox column which needs some osmosis modifications to not bail out on the different schema. See here http://lists.openstreetmap.org/pipermail/dev/2008-September/011640.html for the patch against Osmosis 0.29.
This implementation has some security measurements by not handing out requests but rather return 50x when the data in the database is older than 15 minutes. For this reason the cgi scripts reads the osmosis timestamp.txt. Another safety measure is to not allow an arbitrary number of parallel requests. By this the cgi scripts writes its pid into a directory and counts pids before handling the requests - if above a certain amount it simple returns a 50x. Both number may need tweaking based on the needs and hardware limitation.
First install some dependencys - I guess these are not all - just the ones I remember:
apt-get install ant openjdk-6-jdk openjdk-6-jre openjdk-6-jre-headless
Then unpack osmosis, patch and build it:
unzip osmosis-20080828.zip [...] cd osmosis-0.29 cat ../osmosis-0.29-patch | patch -p1 patching file src/com/bretth/osmosis/core/pgsql/v0_5/impl/ChangeWriter.java patching file src/com/bretth/osmosis/core/pgsql/v0_5/PostgreSqlWriter.java ant [...]
Note: The freshly built osmosis will be inside the build/dist sub-directory. The osmosis.jar in the root directory is distributed with the package and will fail because it is un-patched. Therefore you need to execute:
apt-get install postgresql-8.3 postgresql-8.3-postgis
Follow instructions in /usr/share/doc/postgresql-8.3-postgis/README.Debian.gz on how to enable the GIS extensions in your database.
Then put the osmosis simple shema into your database:
cat osmosis-0.29/script/pgsql_simple_schema.sql | psql osm
But make sure the line concerning the bbox on the ways is not there. Now get the current planet export from http://planet.openstreetmap.org/ Then you may import the initial planet into your postgres:
osmosis --read-xml file=planet-XXXXX.bz2 --write-pgsql user=dbuser database=osm password=dbpass
This will take at least a day until the import finishes. Make sure you use the patched osmosis otherwise the import will basically never finish.
If you experience slow requests after importing the database, you may need to build two extra indexes with the following commands:
create index idx_ways_big on way_nodes using btree (way_id); create index idx_nodes_big on way_nodes using btree (node_id);
These may take an hour or so to complete, please be patient.
Keeping the Database uptodate
To update the database via omsosis I run this script permanently in the background e.g screen session :
while true; do timeout 180 \ osmosis \ --read-change-interval \ workingDirectory=/home/flo/update/ \ --write-pgsql-change \ user=flo \ database=osm \ password=password; sleep 60 done
this will run osmosis every minute which - given the right config - will download the osc changes files from planet.openstreetmap.org and apply the changes to the database.
Here is the osmosis configuration.txt file:
baseUrl=http://planet.openstreetmap.org/minute intervalLength=60 changeFileBeginFormat=yyyyMMddHHmm changeFileEndFormat=yyyyMMddHHmm maxDownloadCount = 15
Alias /api/0.5/map /home/flo/cgi-bin/map.fcgi <Directory "/home/*/cgi-bin/"> Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch AllowOverride None Order deny,allow Allow from all AddHandler cgi-script .cgi AddHandler fcgid-script .fcgi </Directory>
The CGI script is rewritten for mod-apache2-fcgid but should work as a simple cgi with some little modifications too. As the cgi itself is mostly long running the fcgi mod makes no real difference.
The current implementation as of 2008-10-23 can be found here: http://silicon-verl.de/home/flo/projects/osm/roma/
Debian packages used
Right now the ROMA uses Osmosis 0.29 and these Debian/Lenny packages:
Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad) ||/ Name Version Description +++-======================-======================-============================================================ ii apache2 2.2.9-7 Apache HTTP Server metapackage ii libapache2-mod-fcgid 1:2.2-1 an alternative module compat with mod_fastcgi ii libcgi-fast-perl 5.10.0-16 CGI::Fast Perl module ii libdbd-pg-perl 2.8.7-1 Perl DBI driver for the PostgreSQL database server ii libdbi-perl 1.605-1 Perl5 database interface by Tim Bunce ii libfcgi-perl 0.67-2.1+b1 FastCGI Perl module ii libtimedate-perl 1.1600-9 Time and date functions for Perl ii openjdk-6-jre 6b11-6 OpenJDK Java runtime, using Hotspot JIT ii perl 5.10.0-16 Larry Wall's Practical Extraction and Report Language ii postgresql-8.3 8.3.3-1 object-relational SQL database, version 8.3 server ii postgresql-8.3-postgis 1.3.3-3 geographic objects support for PostgreSQL 8.3 ii postgresql-client-8.3 8.3.3-1 front-end programs for PostgreSQL 8.3
Trapi is documented on it's own wiki page.