Osm-conflation-engine

From OpenStreetMap Wiki
Jump to navigation Jump to search
View on OSM Apps Catalog  
osm-conflation-engine
v · d 
Author: Kylenz
License: MIT License
Platform: Web
Status: Active
Version: 3.13.0 (2026-10-22)
Language:
multiple languages
Website: https://osm-nz.github.io/osm-conflation-engine
Source code: osm-nz/osm-conflation-engine GitHub
Programming language: TypeScript

A framework for building automated Conflation-based import projects

Features
Feature Value
Map Display
Display map yes
Map data
?
Source
?
Rotate map
?
3D view
?
Shows website
?
Shows phone number
?
Shows operation hours
?
Routing
?
Navigating
?
Tracking
?
Monitoring
Monitoring
?
Show current track
?
Open existing track no
Altitude diagram
?
Show POD value
?
Satellite view
?
Show live NMEA data
?
Show speed
?
Send current position
?
Editing
Add POIs yes
Edit / Delete POIs yes
Add way yes
Edit geometries yes
Edit arbitrary tags of existing OSM objects yes
Edit relations yes
View notes
?
Create notes
?
Edit notes
?
Work offline
?
Support imagery offset DB
?
Upload to OSM yes
Rendering
?
Accessibility
?

osm-conflation-engine is a software project which helps you to create import projects that run automatically, and use partially-automated Conflation.

For example, it can be used to regularly compare and sync street addresses from a government dataset into OSM. For obvious reasons, there must be a human who reviews all the automated suggestions before they're actually uploaded to OSM.

To use the tool, go to the website: osm-nz.github.io/osm-conflation-engine

Background

There are several limitations with ordinary OpenStreetMap import projects:

  1. They are usually one-off projects, which rely on manual data processing, so the imported data becomes out of data very quickly.
  2. Each region's import projects need to create their own bespoke software or system to review changes before uploading them to OSM. This creates a lot of duplicate work.
  3. Splitting up the work of an import project between multiple people is very difficult, and there's a risk of missing certain areas.

"osm-conflation-engine" tries to fix all these problems. It allows you to create an OSM import project using very minimal code.

  • Instead of a one-off import, you configure a regular scheduled synchronisation. This automatically compares every feature in your dataset with OSM, and flags features which are missing or have the wrong tags.
  • Every import project uses the same unified website to perform the importing, as well as same shared integrations with tools like MapRoulette and RapiD.

It handles all the complicated parts of conflation for you:

  • Setting up a system that runs regularly
  • Publish the results from the comparison
  • Convert the conflation results into a format that can be loaded into OSM editors or MapRoulette.
  • Implementing the matching between OSM features and rows in the source dataset

System Architecture

History

Scalability

Despite being written mostly in TypeScript with minimal Golang, this project has not hit a performance bottleneck (yet).

The largest import project using this tool is the LINZ/Address Import, which has 2.2 million rows, and covers a land area larger than the UK. For that project, the conflation takes about 3 minutes to run.

Tags

This tool does not use any OSM tags itself. That is all defined by the import projects, this tool only provides the infrastructure and website.

Prerequisites

Imports that want to use "osm-conflation-engine" need to meet all these requirements, in addition to Import Guidelines:

  1. Your import needs to use a ref:*=* tag on every OSM feature, such as ref:linz:place_id=*. This is non-negotiable, otherwise it would be extremely slow, unreliable, and computationally expensive to perform automated conflation.
    • if you're importing data that doesn't have distinct features, or doesn't have a clear ID for each feature (like rivers or buildings), then this tool is not for you, sorry.
  2. You need some way of downloading the source data programatically with a script.
  3. You need some basic knowledge of TypeScript and how to use GitHub or another code-hosting platform.

Legacy systems (Public Transport / GTFS)

osm-conflation-engine is a combination of 7 different import projects from New Zealand, which were later unified into a single system. Therefore, some older parts of the system are not fully integrated with the new website.

For example, Public Transport (GTFS) conflation operates quite differently, and is described on a separate wiki page: Osm-conflation-engine/GTFS.

Screenshots

  • The page to select an import project
    The page to select an import project
  • The page to select which task/area you want to review
    The page to select which task/area you want to review
  • The page to select and review specific features
    The page to select and review specific features
  • The page shows metrics and statistics about the import
    The page shows metrics and statistics about the import

Usage

To create an import project using this tool, you only need to define three things:

  1. A script to download the data from your source data
  2. A script which is given 1 OSM feature and 1 row from your source data. The matching is performed automaticaly, so your script just needs to check if the OSM tags are correct for the provided pair.
  3. A JSON configuration file. For example, you can control how matching works, whether geometry should be compared, and how to download the OSM data (Overpass, Postpass, Planet file extracts, etc.)

For a very simple example project, see osm-nz/doc-accommodation-import GitHub.

See the README on GitHub for more details.

Discussion with the community

See Also

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Osm-conflation-engine&oldid=3107449"