Quality assurance

From OpenStreetMap Wiki
(Redirected from QA)
Jump to navigation Jump to search
See also: Accuracy, Completeness

Quality Assurance tools help lead to better quality of OSM data. Often, the tools achieve this by providing a list of bugs in the data, which mappers can then go and fix using editing tools. The bugs are either automatically detected based on rules and data analysis, or the tools provide a means of manually reporting them, or some combination of the two. There are many different ideas for how to do this, and what kind of bugs to focus on, hence many different tools.

OpenStreetMap is often more up-to-date and of a higher quality than other commercial maps when dealing with New and Changed Ways.

It is important to understand that warnings are just warnings, they should be examined and fixed where possible but never at the cost of introducing wrong data. Some validators warnings are generated for valid data. Often in such cases validator code should be fixed, not OSM data. Sometimes false positives are unavoidable.

Bug reporting tools

These are tools that try to highlight and point out parts of the data that are likely wrong. There are also reports from manual quality checks by humans. In both cases, the reported issues may contain false positives and, when editing, care should be taken to ensure their accuracy. However, they can provide a very valuable visualisation of which areas need attention and can help spot and correct errors.

  • Notes
    Notes is a feature of the OpenStreetMap website. Error reports can be added by using the 'Add a note' button in the lower right corner. Notes uses the same concept as OpenStreetBugs, which it has superseded.
  • MapDust
    MapDust is a bug tool operated by skobbler, which is also integrated into their OSM-based turn-by-turn navigation solution, thus originally focusing on navigation-relevant bugs but now serving as a general bug tool. As many of its users (end users!) may not be aware of OpenStreetMap at all, there are also quite a few non-relevant error reports included, but it does contain many good reports that are not covered by other tools. If a bug is determined to be caused by OSM data, an OSM editor can fix the data and mark the bug as resolved. Currently it is available in English, German, French, Italian, Spanish and Romanian.

Error detection tools

Error Detecting Tools check the OSM data for potential data errors, inaccuracy or sparsely mapped places. Users should check if these structures are really wrong (false positives usually occur and there are not really mapping rules which are set in stone) and correct the data for a continuously rising data quality.

Comparison of some of the following tools
Tool Coverage Error types Display mode Fix suggestion Downloadable API Correction guide Developement
Keep Right World Many (50+) Marker map no yes yes German only Inactive (last commit 2017)
Osmose World Many (200+) Marker map yes yes yes yes Active
JOSM/Validator Local Many List + map yes yes N/A For some problems Active
OSM Inspector World/Partial Many Rendered map no yes N/A no Active
MapRoulette World/Partial Many (10+) One feature at time yes yes yes no Active
BRouter Suspects World Car Routing List or Osmoscope no no no no Active
Nominatim QA World Addressing, places Marker map yes
  • Keep Right
    Keep Right (keepright.at) shows a large number of automatically detected potential errors on a map or in list form. It has a system for reporting false positives and for labelling a bug as fixed. It has rules to automatically detect the following error types: non-closed areas, dead-ended one-ways, almost junctions, deprecated tags, missing tags, bridges/tunnels without layer (careful - not always an error), motorways without ref, places of worship without religion, POIs without name, ways without nodes, floating islands, un-tagged railway crossings, wrongly-used railway crossing tag, objects with FIXME=* tags, and relations without type.
    The bottom-right corner of the interface has links to data outputs, including GeoJSON.
    There is a documentation on how to solve keep-right-errors, currently only in German language: DE:Keep Right Users Guide though here is an auto-translation into English.
    Possible downsides: Keep Right shows tens of thousands of errors, including many that have little impact for most data uses (e.g. a stream intersecting a highway: technically some feature is needed here (a bridge, a culvert or a ford). Keep Right uses flawed assumptions about layers of bridges and tunnels sometimes leading to false or misleading error messages (discussion).
  • Osmose
    Osmose (osmose.openstreetmap.fr) is a tool similar to Keep Right but offers even more error types. Obviously, this also includes smaller issues, like minor imprecisions or missing information that should be present in an ideal map.
  • JOSM Validator
    JOSM/Validator. A feature of JOSM that checks data loaded into the editor, highlights errors and warnings, and can (on request) perform some automatic fixes. By default, it checks all objects modified in that session (and also reports errors if the error wasn't introduced by the uploader, but merely touched), but the validator can also perform complete validations on the downloaded data.
  • The Gary68 tools
    Gary68 has created a whole bunch of quality assurance tools. The tools are currently not maintained. Most of the tools are written in Perl (scripts for your computer – no web tools) and the source code of all Gary68 tools is published online. See also Category:gary68.
    • WayCheck is a program which checks the OSM data (file) for open ends and crossings. The found positions must be evaluated by a user and he or she can correct the data in JOSM or Potlatch or post a mark in OpenStreetBugs. The WayCheck page lists reports generated by the program and any user is encouraged to run reports and correct errors listed on the page and its links. This is especially useful for the quality of routing related data.
    • MotorwayCheck lists problems related to highway=motorway. Page also shows reports.
    • osmdiff is a program which compares two OSM files and documents the changes in list and map form. Some reports can be seen here: osmdiff reports.
    • unmappedplaces finds a list of potentially unmapped places. These are reported on the wiki page. The query can be run for other countries as well.
    • Mapping Quality is the successor of unmappedplaces. See the wiki page for the results. You get automatic street directory, KPIs (numbers) per place, maps that show status and CSV files for further processing.
    • SomeChecks performs some checks: One-way check, Roundabout check, Double-node check (ways), Area Check.
    • Relation Diff - Differences in relations
    • Relation Check
  • OSM Inspector
    OSM Inspector (short: OSMI) has several views for specific mapping needs.
  • TIGER Edited Map
    TIGER Edited Map ([1]) - Map showing which areas of the U.S. have been edited since the TIGER import.
  • Coarse Highway
    Ragged highways with few points (blog entry): Coarse Highways.
  • OS Locator Musical Chairs
    OS Locator Musical Chairs ([2]) attempts to intelligently match OSM streets in GB against the OS Locator streetname database and show a browsable interface of the results.
  • Map of Turn Restrictions by Zartbitter
    ahorn.lima-city.de/tr/ shows turn restrictions and has separate overlays for errors and warnings showing potential problems with turn restrictions. See also Relation:restriction#Tools for more tools.
  • Restriction Analyzer by MorbZ
    OSM Restriction Validator shows unnecessary restrictions and restrictions that block access to streets.
  • US Interstate refs
    US interstate refs List of ref numbers and corresponding OSM interstates.
  • layers.openstreetmap.fr
    FR:Servers/layers.openstreetmap.fr provides a slippy map with image layers to help people find possible errors in the data with worldwide coverage. The service provides:>
    • layers with roads/street …
      • having no name
      • no ref
      • no one-way
      • drawing fixmes and notes texts.
    • This tool also generates half-transparent coloured layers showing administrative areas by admin_level, as well as other "political" areas (used in France for mapping electoral subdivisions, as well as intercommunal structures of cooperation that do not fit the hierarchy of admin levels, but used as well in other countries for similar non hierarchical administrative, judiciary and electoral boundaries). These coloured layers allow searching easily for missing/broken/incomplete/unclosed boundaries as well as overlapping/duplicate boundaries of the same type, or boundaries that are not tagged correctly to be found by querying the OSM database with simple filters (for example missing tag "boundary=administrative", missing or incorrect value in "admin_level=*").
    This tool shows polygons which present a problem for Nominatim. These ‘broken’ polygons could be where the interior of a multipolygon is disconnected or, where a polygon intersects itself.
  • MapRoulette
    MapRoulette (MapRoulette) is a customizable web application to crowd-source the fixing of specific mapping errors by randomly displaying an item of a specific type in potential need of fixing.
  • netzwolf's opening_hours & co. map
    [3] - Check Opening_hours, Service_times...
  • Routino error log
    Routino a router for OSM data can generate an error log file (when run with the --errorlog option) that lists problems found that could be data errors or Routino's limited understanding of the data and which may affect the ability to calculate valid routes. Since the purpose of the Routino error log is to find data that causes problems for Routino it does not mean that all reported items are data errors.
  • Bicycle tags map
    Bicycle tags map is a web map that shows bicycle related tags. The goal of this web map with bicycle tags is rendering bicycle related tags that are not (properly) rendered on OSM’s 'Mapnik' map or on the cyclemap version. It can be handy for detecting errors in OSM.
  • Max Speed Overpass
    Max Speed Overpass is a web map that shows maximum speeds. The goal of this web map is to show max speed and edit this from this map using your favorite OSM editor. The data shown is always up to date because it uses overpass query.
  • CheckAutopista
    CheckAutopista [4] is a web map for checking information on freeways. It allows to check if maxspeed, lanes and name are used on the ways and it checks if the exits have their ref and their exit_to (or destination). Also it shows the freeway's service and rest areas and it's tollbooths.
  • Relation Analyzer
    The Relation Analyzer (http://ra.osmsurround.org/) helps to view several quality-assurance-related aspects of relations: (amongst others) relation on a map with segments' start and end marked, warning if the relation has gaps accompanied by a list of the segments if there are gaps, a height profile, type of ways included, tags.
  • qa.poole.ch
    qa.poole.ch is a tool to show streets with no names (respects noname=*), noaddress=* and access=yes tags.
    The layers are updated very frequently, after a minute you should be able to see your edits reflected.
  • Improve OSM
    This set of tools from Telenav helps fix OSM based on the vast amounts of GPS data Telenav is gathering from their GPS Navigation & Maps - Scout App (blog post).
    This includes identification of potential missing roads, oneway streets and turn retrictions.
    The tools are available as a website and a JOSM plugin.
  • OSM-Sidewalker by Mapbox
    OSM-Sidewalker by Mapbox shows an overlay of ways which appear to be a sidewalk but are missing the footway=sidewalk tag. The tool was introduced in a blog post.
  • Unmapped places
    Unmapped places is a bug tool done by neis-one.org which marks places without roads.
  • Errors in Alternativas libres
    In download section of Alternativas libres you can find a text file that marks the errors detected when they do their maps for Garmin. It is divided by countries and information is updated daily.
  • OSMsuspects!
    OSMsuspects! (Germany only) checks for wrong addr:city=*, addr:street=*, addr:postcode=* tags by comparing to OSM boundaries and public governmental data. Also checks if at least street and number is available on a address object. More info (german only).
  • Is OSM up-to-date?
    Is OSM up-to-date looks for old and potentially outdated nodes and ways.
  • BRouter Suspect Manager
    BRouter Suspects are potential problems that affect the connectivity of the road network. The detection is based on BRouter's routing engine using a car profile, and mostly focuses on dead-ends or dead-starts on highway=tertiary an higher, plus some other detection patterns related to blocked roads touching the routable network. In addition to marking a suspect as fixed or false-positive, the suspect manager also allows hiding suspects for a given time, which makes it also applicable for the tracking of temporary construction edits or more complicated issues involving changeset discussions or OSM notes. Also check the Osmoscope view on the BRouter Suspects.
  • housenumbervalidators
  • Osmoscope
  • OSM POI Analyzer (OPA) is an online platform to assess and measure the positional plausibility of POIs in OSM. Currently only 6 cities and 15 Tags in the world are covered by OSM POI Analyzer, but developers are open to suggestions to add more cities and tags to this platform. The platform is using a recommendation system which is based on association rule mining. Distance-based and topological rules are generated for each city and tag (object) and saved in the database. How it works? Users can click on map and try to add an objects. For example choose Paris city and ATM object from right side menu and then click in a corner of street in Paris. The analyzer needs 1 to 2 seconds to process and then brings up the recommendation score with some additional results. However this is experimental and the new POI will not be added to OSM database.This allows users freely navigate, click on places and measure the accuracy of recommendations in this system. For example you can try to click inside jungles and rivers and try to add a fuel station. This platform is developed at the end of Phd work by Mr. Alireza Kashian at University of Melbourne to improve concepts around association rule mining in OSM. The platform is currently hosted and funded by donations. Publications about methodology and how the whole system works is accessible in publication page. There are plans to cover almost 100 major cities of world by Mid 2020.
  • staty
    staty is a tool to assess the quality of public transport station tagging in OSM. It checks for suspicious station names and provides suggestions to fix names or group similar stations into public_transport=stop_area or public_transport=stop_area_group relations.
  • Subway Preprocessor
    Subway Preprocessor helps to maintain the quality of subways and other rapid transit networks. It reveals logical and topological errors like a stop position being not connected to a station via public_transport=stop_area relation, or incorrect order of stops in a route relation. It also allows to view a subway scheme on the map. Related links are:
https://github.com/alexey-zakharenkov/subways — active repository fork
https://maps.mail.ru/osm/tools/subways/latest/ — weekly refreshed validation results
https://maps.mail.ru/osm/tools/subways/ — history of recent validations
https://maps.mail.ru/osm/tools/subways/validate/ — validator trigger (please do not overuse)
https://maps.mail.ru/osm/tools/subways/latest/mapsme.json — file suitable for routing over validated rapid transit systems, including transfers.


Visualization tools

  • Pedestrian overlay
    [5] – A map of pedestrian/walking data; useful when checking coverage or connectivity of pedestrian mapping.
    It aims to show pedestrian mapping coverage (how many of the pedestrian-taggable elements are tagged with pedestrian data); this may be useful to see how well an area is mapped for pedestrians.
    It also aims to give an idea of connectivity (how well elements are connected, i.e. whether pedestrian elements have nodes connected to each other via nodes, for example, sometimes pedestrian elements are mapped without showing how they connect, for example a path which connects to a sidewalk may not yet have a node connecting one another); this information may be helpful for those working on routing.
  • OSRM debug option
    [6] - The debug option of the OSRM demo page provides a map overlay of the highways' speeds calculated by OSRM.
  • OSM Lane Visualizer
    [7] OSM Lane Visualizer is a tool to show several attributes of a highway, selected by a overpass query or by relation id.
  • Parking lanes viewer from OpenStreetMap
    [8] Parking lanes viewer from OpenStreetMap is a map to show attributes of a highway, such as parking:lane=*, the source code is located under: Blacktocat.svg zlant/parking-lanes.
  • Bus lanes viewer from OpenStreetMap
    [9] Bus lanes viewer from OpenStreetMap is a map to show attributes of a highway, such as lanes:bus=* or lanes:psv=*, the source code is located under: Blacktocat.svg zlant/bus-lanes.

Monitoring tools

In addition to the above error and bug reporting tools, there are a number of tools that allow you to spot erroneous changes and edits. For example if you are very familiar with an area and have thoroughly mapped it, you might want to follow all changes and verify that no unintended damage or vandalism has happened in the area. The Planet History includes all changes that have been made, but might not be as easy to use.

In case that a change is bad (by 'mistake' or vandalism) you can rollback the change.

The following and other tools also listed on the List of OSM based Services page. For approaches to automatically detect suspicious changesets see Detect Vandalism.

  • osm.org
    API v0.6 introduced the concept of changeset. This is a group of edits made by particular user, with a comment. Some new displays have been developed for the web front end:
    • 'history' display showing recently closed changesets possibly affecting (bounding box overlap) the currently shown map area
    • 'User edits' display showing recently closed changesets by a particular user.
  • Real-time change following
    Currently nearly the same content: List of OSM based Services#Live/real-time edits to OSM data
    • LiveEditMapViewerJ is a Java program that allows you to follow changes as they happen on a world map.
    • OSMZmiany is based on LiveMapViewerJ but has some additional features.
  • History Browser
    virtuelle-loipe's History Browser with diff tool for single nodes/ways/relations (side by side comparison of two versions).
  • magOSM - tracking changes
    magOSM - tracking changes shows changes grouped by thematic and change type on a heat map and give you access to the change detail with several links to other quality assurance tools. Available only on metropolitan France.
  • Mapki's Deep Diff
    http://osm.mapki.com/history/ - diff tool for single nodes/ways/relations. Showing all versions in a table. A bookmarklet is available.
  • OSM Deep History
    https://osmlab.github.io/osm-deep-history/ - diff tool for single nodes/ways/relations (but without nodes of ways and without members of relations). Showing all versions in a table. A bookmarklet is available.
  • OSM Visual History
    https://aleung.github.io/osm-visual-history/ - visualized diff tool for single nodes/ways/relations. Showing all versions position on map and tags in a table.
  • OSM History Viewer (by PeWu)
    OSM History Viewer (by PeWu) (pewu.github.io/osm-history) - diff tool for single nodes/ways/relations. Showing only the colour-coded changes in a list. A bookmarklet is available on the page.
  • OSM History Viewer
    OSM History Viewer (OSMHV) shows modifications by single changesets in a visual way, highlighting object positions before and after a given changeset and tag changes. See also: the tool achavi has a similar feature.
  • OSM Aware
    OSM Aware is a program (only Python source code, no web service anymore) to create KML files of OSM change positions.
  • UserActivity
    UserActivity Statistics and maps of user activity in two OSM files. Possibly detect vandalism... Reports (for Germany) generated periodically.
  • WhoDidIt
    The WhoDidIt: OpenStreetMap Changeset Analyzer analyses what was changed in your area. You may need to select "month" or even "half a year" for the "age" parameter to see edits; display could take some seconds. Features an RSS feed to get informed about new changes.
    WTFPL licensed. At least two alternative versions exist. Longer description.
    Live implementations:
  • Latest OSM Edits per Tile
    https://resultmaps.neis-one.org/osm-change-tiles#13/50.0042/8.2912 - analyses latest OpenStreetMap edits per tile. Features an RSS feed to get informed about new changes.
  • RSS History Filter
    RSS History Filter filters out large changesets from RSS feeds generated from the OpenStreetMap history tab
  • OSM controltool
    https://wheregroup.com/produkte/osmct/ ­– commercial product by Wheregroup that allows to monitor changes in freely definable polygons. Only small demo version available without registration.
  • tyrasd's RSS feed link creator
    QA RSS feed links for several services can be generated for a self-defined boundingbox at tyrasd.github.io/osm-qa-feeds.
  • tyrasd's Latest Changes service
    at https://tyrasd.github.io/latest-changes/ shows all recent changesets (within a week, day or month) which touched the current map area. Changed objects are highlighted in the map. The object highlight is changing colour (to less intense) with the age of the relevant changeset. Deleted objects (and old versions of modified objects) are shown as faint outlines.
  • achavi
    achavi (Augmented Change Viewer) visualizes OpenStreetMap changes based on the Overpass API Augmented Delta (adiff) query. Supports continuous monitoring of an area of interest and visualizing single changesets (with some limitations).
  • Overpass attic queries
    With Overpass' attic queries you can query the OSM data based on time – example for objects which existed at a specific point in time or that were edited during a period of time.
  • OpenStreetMap Analytic Difference Engine
    The OpenStreetMap Analytic Difference Engine (also called "OSM Analytic Tracker" or OSMAT for short) continually analyses activity in a given region (depending on the level of activity, ranging up to small countries) and presents tag changes in fairly easy to digest lists, allowing experienced contributors to spot and react to mapping mistakes in their region within minutes. Also allows opening "diffmaps" (similar to achavi etc) for quick overviews of geometric changes. User Cascafico on his blog documents how to set up an instance on an OrangePI PC Singleboard computer.
  • OSMarelmon - The OSM Relation Monitor
    OSMarelmon monitors relations of OpenStreetMap. Monitored relations are periodically checked and the changes are distributed via RSS feeds.
    Helper tools like IFTTT might be useful to convert (possibly a subset of) e.g. QA RSS feed entries into a mail notification.
  • Where are the new OSM Contributors?
    Pascal's resultsmaps contain a "Where are the new OpenStreetMap Contributors?" feature. You can spot new contributors who have edited in a specific area (may be useful to check their edits for big errors, offer help, send welcome messages and spot vandalism). Create a RSS feed.
  • OSMCHA - OSM Changeset Analyzer
    OSMCha features an extensive changeset filtering including detecting suspicious edits.
  • Osmlab Changeset-map - OSM Changeset Viewer
    Osmlab Changeset-map Example CS Visualise a changeset on a map. Github
  • osm-suspicious
    osm-suspicious by Pascal Neis for finding suspicious changesets.
  • OSM Hall Monitor
    OSM Hall Monitor With OSM Hall Monitor, you can track edits made by specified users, made to specified objects, or made with certain tags. You can also enable notifications to receive emails when any of the flagged people or items are edited. Basic functionality for suspicious changeset monitoring is also included.
  • OSM Edits Made With MAPS.ME
    [10] With this tool you can see edits made with MAPS.ME. You can also see by country, by user, etc. open the objects in JOSM or Level0. etc. .

Assistant tools

  • Traffic Sign Tool
    Traffic Sign Tool / Verkehrszeichen_Tool (de) - Assistant to get the right tags for German road signs.
  • Road lists
    Users in Germany requested different official lists: Straßenverzeichnis.
  • CheckTheMonuments
    Check The Monuments! More Quality for the Key Monument.
  • Overpass turbo
    Overpass API and the interface Overpass turbo lets you easily create custom queries which can be used to detect potential errors. The results get shown on a map with object detail links in popups and can be easily opened in JOSM. The example page lists some quality assurance uses.
  • (Missing) Maxheight map
    The Maxheight Map (aka OSM Truck QA Map) (website) is a Overpass API-based browser tool which helps you check and improve truck relevant features in OSM. Besides identifying missing maximum height tags under (railway) bridges and tunnels, you can also validate existing tagging for maximum height, width, length and weight amongst others.
  • OSM Quality Assurance Editor (by osmsurround)
    The OSM Quality Assurance Editor (website) helps to highlight (via Overpass API) missing data in several categories (tracks without tracktype tag, tracks without surface tag, streets without any sidewalk tag, ways without incline tag, ways without surface tag, ways without smoothness tag, buildings without address, residential without name) and allows to directly edit the objects via a built-in online editor.
  • Public Transport Network Analysis (PTNA)
    PTNA finds and aggregate information about public transportation lines mapped in OSM. Additionally a quality assurance is integrated. GTFS data is visualized and tags for route relations recommended.

Tag statistics

To compare the use of a tag or to see typos, use these statistical tools. See also: Statistics.

  • Taginfo
    Taginfo is a system for finding and aggregating information about OSM tags and making it browsable and searchable.
  • OSM Tag History
    OSM Tag History displays the number of times a key or a key-value pair has been used as a function of time on a visual graph
  • ohsome
    The ohsome dashboard and API allow to perform in-depth analysis of the development of OSM data.


External compares

See also: Research

See also