Overpass API/Augmented Diffs
The augmented diffs extend the ordinary minute diffs. They make it possible to keep even specialized databases up to date minutely. They are based on an idea by Matt Amos, refreshed by Frederik in a talk in German.
Augmented diff API endpoint: http://overpass-api.de/api/augmented_diff (stable endpoint with long term support)
Experimental augmented diffs (discontinued with July 2014): http://overpass-api.de/augmented_diffs/
The augmented diffs contain three kinds of data
- The new data that comprises the usual minute diffs
- The old data that is either replaced by new data or explicitly deleted
- Meta data for deleted objects
The ways and relations are in addition enhanced by their geometry.
This allows in particular the following two applications
- Proper minute updates on datasets restricted by a bounding box
- A wayback mechanism: You can start with a planet file or other database and walk back instead of forward through time
While the use of the former is immediately clear, the second needs some explaining words: A couple of examples from the talks above like the counting application need to know what is deleted to decrease the counter.
Each diff is structured in three sections, one for each type of element. Within a section, the elements are ordered by id. Each element only appears its state at the beginning and the state at the end of the minute. Shorter lived versions are omitted.
Every element is considered as having changed when its meta date has been updated or its tags have changed. In addition, nodes are considered to have changed when their coordinates have changed. Ways are considered as changed when their members have changed or one of its members has changed its coordinate. Relations are considered as changed when their members have changed or one of its way members has changed by its members or coordinates or one of the node members has changed its coordinate. By contrast, changes on members of type relation don't propagate to their parent relations.
In the preamble, the file contains the end date of the diff.
<?xml version="1.0" encoding="UTF-8"?> <osm version="0.6" generator="Overpass API"> <note>... </note> <meta osm_base="2012-09-20T20\:24\:02Z"/> <!-- Elements are ordered as: nodes first, then ways, then relations. Within each class of elements they are ordered by id --> </osm>
Each element is wrapped in an action of type delete, modify, or create, depending on whether the element is deleted, changed, or newly created.
<action type="delete" ...> <old> <!-- OSM element --> </old> <new> <!-- OSM element --> </new> </action>
An object is noted as "deleted" if it has been deleted in the Augmented Diff. Because this produces a new version, there exists also a "new" section. This contains an empty OSM element with marker "visible=false" and the meta data of the deletion.
<action type="modify" ...> <old> <!-- OSM element --> </old> <new> <!-- OSM element --> </new> </action>
Same as "delete". Here, the new version is the proper new version of the element. Please note that for ways and relations old and new version could have the same version number. This happens when the element itself is unchanged but a member element has changed and thus also the geometry of the element itself.
<action type="create" ...> <!-- OSM element --> </action>
The "create" action only contains one version, the new version.
All way and relation elements contain a "bounds" element. This elements tells you the enclosing bounding box.
All nd elements inside of way elements contain additional lat and lon attributes:
<way id="..." ...> <bounds ... /> <tag ... /> ... <nd ref="..." lat="..." lon="..." /> <nd ref="..." lat="..." lon="..." /> <nd ref="..." lat="..." lon="..." /> ... </way>
All members of type node of relations also contain additional "lat" and "lon" attributes:
<rel id="..." ...> <bounds ... /> <tag ... /> ... <member type="node" role="" ref="..." lat="..." lon="..." /> <member type="node" role="" ref="..." lat="..." lon="..." /> <member type="node" role="" ref="..." lat="..." lon="..." /> ... </rel>
Members of type way repeat their geometry in a sequence of "nd" subelements:
<rel id="..." ...> <bounds ... /> <tag ... /> ... <member type="way" role="" ref="..."> <nd lat="..." lon="..."/> <nd lat="..." lon="..."/> <nd lat="..." lon="..."/> </member> ... </rel>
Time slices and numbering
The augmented diffs are numbered with consecutive numbers. The zero point is the license change, i.e.
will tell you that it is going to fetch the difference between "2012-09-12T06:55:00Z" and "2012-09-12T06:56:00Z". From that date on, the counter is increased once per minute.
You can get the last available augmented diff via the API call
The augmented_diff API call
On overpass-api.de exists an API that allows to filter the Augmented Diffs already on the server. The base URL is
It accepts the following three arguments:
- id= is mandatory and is the id of the Augmented Diff to process
- bbox= is optional and is a min_lon,min_lat,max_lon,max_lat limited bounding box of the area of interest
Software and Services using Augmented Diffs:
- osmconvert (since 0.7E): The program can now read Augmented Diffs and convert them to .osc. It also can process them and update an existing .osm or .pbf. All this is highly experimental, you should expect one or two bugs... :-)
- Osm-watch: osm-watch is an OSM contributions almost real-time monitoring tool with filters based alerts that you can created and receive by RSS or email.
- achavi: Augmented Change Viewer - visualizes updates to OpenStreetMap
- Show Me The Way on OSMLab