Talk:Overpass API/Installation

From OpenStreetMap Wiki
Jump to: navigation, search

Additional functionality like management of areas aren't covered yet.

Arrr too bad ! Isn't there even the slightest tiny bit of documentation to explain how the <area-query> could be enable ? Searching the code lead me to /src/overpass_api/statements/area_query.c but I'm unable to answer : "what tool should I use to build those area things after or during the initial import" Some scripts in the osm-3s_v0.6.99 seams to suggest something like :

  • ./dispatcher --areas

To enable areas at dispatcher level, but what about imports and diffs ? Well, at least it's worth trying that on a test db ;-) sletuffe 01:35, 18 October 2012 (BST)

I'm sorry for the entire hassle.
Why whould you be sorry ? thanks for everything ! sletuffe 10:33, 18 October 2012 (BST)

First, the technical things: Run

nohup ./dispatcher --areas --db-dir=$DB_DIR &

the same way as you do with

nohup ./dispatcher --osm-base --meta --db-dir=$DB_DIR &

This makes it possible to both read and write areas. Nohup is not mandatory, but surely useful.

However, there aren't any areas yet. To get these areas, create a subdirectory rules in $DB_DIR and copy rules/areas.osm3s into this subdirectory. Please run then

nohup ./rules_loop.sh $DB_DIR &

This starts a long lasting process that reads from osm_base and writes into areas in one huge transaction. Now wait 4 to 24 hours, depending on the hardware. The areas get available once you see the entry update finished in the log file $DB_DIR/rules_loop.log.

Now, the caveats:

  • The file size of the main database may significantly increase with area creation, from about 85 GB to about 145 GB, at maximum 170 GB if the area creation is very slow: as the area creation happens in one large database transaction, the database state from the beginning is conserved in a shadow copy that grows when the minute diffs get applied to the main database. The additional file size is not the area related files, but manifests as a growing main database.
  • The area creating process may consume so much resources that it slows down the queries by regular users. To avoid this, use renice and ionice (if available) to give the process a lower priority. On my server, the area creation is set to priority 19 resp. -c 2 -n 7.
  • Some bugs around the area feature popped up only very recently, and I'm sure there are still undetected bugs. So please report all bugs, but be prepared to update to the newest version (or to patch the relevant changes into your code copy).

You can observe progress in the associated nohup.out: the daemon creates rougly every 15 seconds a progress status line.

It is also possible to change the contents of areas.osm3s. This is a proper script in the XML syntax. Unfortunately, you must use the XML syntax here because it is not yet ported to the Overpass QL syntax.

Now some background:

The area feature has been present in the Overpass API since 2009, but in contrast to the more or less sophisticated search operators, it has rarely been used, at least as observations from both log files and questions by eMail indicate. For that reason, the feature has never received much care.

However, as the more urgent problems are solved now and a multiple use cases popped up for area search (including city excerpts on demand and search for a street name and postal code), the area feature gets now a major overhaul. In the end, a query like

  area[postal_code=42119];(way(area)[name="Kronprinzenallee"];>;);out;

will find the street named "Kronprinzenallee" in the area with postal code "42119" (or so, not sure about the tag name). The final query needs however reference to the country. I expect to be done by mid-November. Searching an area by name works already in the deployed code, e.g.

  area[name="Grenoble"];out;

already works in the latest git version and on the server overpass-api.de, but is younger than the latest tested version. -- Roland