LINZ/Address Import

From OpenStreetMap Wiki
Jump to navigation Jump to search
LINZ Main Page LINZ Building Import LINZ Address Import Missing Streets LINZ Place Name Import

This page documents the import of street address data from Toitū Te Whenua Land Information New Zealand (LINZ) into OSM.

  • This project first ran between 2017-2018, when almost two million addresses were imported.
  • In 2021 the project resumed, to import new addresses, and setup an automated system for regular address Conflation.
  • In 2026 the automated conflation code was migrated to the standardised Osm-conflation-engine system.

2017-2018 import complete

The import described below was completed in June 2018 with nearly two million addresses imported. (Note that "complete" doesn't mean that every address was imported, just the process ended)

The import is summarised in this spreadsheet

The source data contained some sets of addresses with many addresses at exactly the same coordinate e.g. apartments or flats. Coincident points mustn't be added to OSM so either the points were manually distributed in space, or they were deleted, perhaps replace with a single address e.g. "1/3 A Street" to "100/3 A Street" replaced with "100 A Street"

The source data was split using the name of city/suburb or hamlet. During import, it was found that there are some hamlets with identical names in different parts of the country: Aramoana, Awatuna, Belmont, Blue Mountains, Clifton, Dalefield, Karamu, Kinloch, Longbush, Matahiwi, Muriwai, Ngahape, Otara, Owhiro, Purangi, Tokanui, Waikawau, Woodside. Any future import/update should consider this.

2021 import continuation

A screenshot of the 2021 import tool (a fork of RapiD), showing the UI to review a feature
A screenshot of the 2021 import tool, showing 42 addresses to be added, 1 to be deleted, and 1 building requiring an edit
A screenshot of the 2021 import tool showing an address that needs to be moved

2021 Goals:

  • To import the 161,000 missing addresses for the New Zealand mainland (and the outer islands, where data exists)
  • To fix typos and other issues with 17,000 existing addresses in OSM that were added in 2017.
  • To add the ref:linz:address_id=* tag to 40,000 addresses where the tag was not added during the initial import, or where it was removed when address points were manually merged with buildings
  • To delete 11,000 addresses that were imported but since deleted by LINZ. See the #Address Deletion section below
  • To set up a system that is simple and efficient, which allows OSM street address data to be regularly conflated with LINZ's data

Source data

Data sourced from LINZ simple street address layer NZ Street Address

Structure of data is documented in the Data dictionary. Details are shown in the #Notes section at the bottom of this page.

The source code is available from osm-nz/linz-address-import GitHub (2021 import). The 2017 import code is no longer available[dead link].

How it works

A script runs regularly which compares the latest LINZ Street Address data with the latest OSM data for the New Zealand mainland. This script requires at least 4 GB of RAM and takes approximately 6 minutes to execute. This script automatically runs weekly using GitHub Actions (a free service), and the generated files are stored in a free CDN.

This script conflates the street address data from LINZ with OSM and categorizes each address into one of the categories in the table below, depending on what changes are required.

This data is made available for anyone to add/update/move/delete. Between 2017 and 2018, JOSM was used. Between 2021 and 2025, we used a fork of the RapiD editor designed for this particular import. Since 2026, we use the standardised osm-conflation-engine system.

How do I contribute?

Go to the homepage of Osm-conflation-engine, where you can find this import project, and start contributing.

Conversion and Tagging

The tagging scheme used is the same as the original 2017 import:

LINZ Field Name OSM Tag
address_id ref:linz:address_id=*
full_address_number addr:housenumber=*
and alt_addr:housenumber=* if the property has an 'Alternate Address'. E.g. 2 or 3A or 4/56B. This follows NZ addressing conventions for unit number prefix.
full_road_name addr:street=*
suburb_locality addr:suburb=*. Historically, addr:hamlet=* was used for rural addresses, until the LINZ dataset stopped distinguishing rural vs urban[1]
town_city addr:city=* if urban, does not exist for rural addresses
water_name If present, addr:type=water will be added.

Other tags:

  • building:flats=* - added if multiple flats are stacked[2] together.
  • check_date=* - added by the editor if an importer ignores the suggestions from the conflation service. Addresses with a check_date will be ignored by the conflation process if the date is less than a year ago.
  • alt_addr:housenumber=* - added to addresses that have an "alternative address", for example if 123A and 1/123 are both valid house numbers for the same place.
  • linz:stack=no - mappers can temporarily add this tag to a stacked[2] address to tell the system to delete the stacked node and replace it with individual nodes for each appartment. This will happen next time the sync runs.

Changeset keys

Keys on each changeset changeset:

Potential issues

This table will be updated as the project progresses.

Issue Mitigation
Duplicate data being imported The fork of RapiD has an added feature to prevent duplicate addresses being imported based on the ref:linz:address_id=* tag. This conflation happens in real-time, in the browser
Multiple people editing the same dataset at the same time Users will be presented with a warning if someone else is/was editing that dataset in the last hour

Filtering

Identifying Duplicates

Each address in the LINZ data is processed, and categorized as follows:

ID Status Status Description Action
1 PERFECT An address node already exists in OSM with the same ref:linz:address_id=*, and the data is correct none
2 EXISTS_BUT_WRONG_DATA An address node exists in OSM with the same ref:linz:address_id=*, BUT the data in the OSM node doesn't match LINZ's data Anyone can use the fork of RapiD to edit this data
3 EXISTS_BUT_NO_LINZ_REF There is no OSM address node with a matching ref:linz:address_id=*, BUT there is an OSM address node with the same data, just no ref:linz:address_id=*
4 MULTIPLE_EXIST_BUT_NO_LINZ_REF There are multiple address nodes for this address, none of which have a ref:linz:address_id=*
5 MULTIPLE_EXIST There are two different nodes in OSM with the same ref:linz:address_id=* Requires manual action for complex situations
6 EXISTS_BUT_LOCATION_WRONG The address exists in OSM, with the ref:linz:address_id=* tag but the coördinates are totally wrong (>500metres away from the gazetted location) Anyone can use the fork of RapiD to add, move, or delete this data
7 TOTALLY_MISSING Address doesn't exist in OSM at all
8 NEEDS_DELETE The address was imported from LINZ, but since then LINZ has deleted the address (see #Address Deletion)
9 NEEDS_DELETE_NON_TRIVIAL The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has added business/POI information to the address node (e.g. name=*, shop=*, craft=*). See #Address Deletion. Requires manual action
10 CORRUPT The address in OSM contains multiple ref:linz:address_id=*, separated by a semicolon. Anyone can use the fork of RapiD to edit this data
11 LINZ_REF_CHANGED LINZ has changed the value of ref:linz:address_id=*. Effectively a delete and a re-create with a new ID.
12 obsolete
13 COULD_BE_STACKED Addresses in OSM that are PERFECT, but the flats/units could be stacked if this is desired. Stacked means the many individual nodes for each flat in an apartment complex are replaced with a single node for the whole apartment complex. More Info. none, just for statistics
14 NEEDS_DELETE_ON_BUILDING Similar to #9. The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has merged the address node into a building. We won't delete the building, but we will remove the address tags from it. See #Address Deletion. Anyone can use the fork of RapiD to edit this data
15 REPLACED_BY_BUILDING The address exists twice in OSM: once on an imported node, and once on a building. We will merge the node into the building. This is usually caused by StreetComplete users.

Non Duplicates

Approximately 3% of NZ addresses are already in OSM in some form (40K/2M). The remaining 97% will be new.

In the first pass, potential duplicates will be identified, and saved as a separate dataset for later processing. The remaining non-duplicates will be uploaded in batches.

Batch membership will be determined by town_city & suburb_locality in common.

A quick delve into the data gives

  • Water addresses 160, Road 1.9 million
  • Localities (town_city = NULL) 1930 distinct. 1 to 2800 items per locality. 25 localities with > 1000 addresses
  • town + suburb 1182 distinct, 258 of which have town=suburb. 645 have <1K addresses, 907 < 2K, 6 > 10K

Using the above distinction would give about 3100 separate batches.

An osmchange file will be generated for each place. After review, each will be uploaded as a separate changeset.

Address Deletion

There are cases when an address was imported into OSM from LINZ, and then deleted by LINZ. Part of this proposal is to delete previously imported addresses in OSM that have been deleted by LINZ. This is possible using the fork of RapiD.

Example: In the following image, address 14 Subdivision Road has been deleted because it no longer exists. Five new properties now occupy the former lot including a new road.

An example of a property that was subdivided, and the original address (No. 14) was deleted.

Out of scope

  • Dealing with duplicate addresses in OSM. If one address has a ref:linz:address_id=*, this project will normally ignore any other identical addresses. There is one exception where this is automatically fixed, see REPLACED_BY_BUILDING above.

Software Tools

During testing, node a single node was uploaded to OSM and then deleted.

Notes

Details of source database

Name Data Type Length Precision Scale Example Description
address_id integer 32 0 505588 AIMS unique identifier for an address.
change_id integer 32 0 1304726 AIMS unique identifier for the address version.
address_type varchar 20 Road The type of address. Includes: Road and Water.
unit_value varchar 70 Alpha numeric value for a unit
address_number integer 32 0 1 Address number
address_number_suffix varchar A Alpha numeric characters that may follow the address number.
address_number_high integer 32 0 High address number of a ranged address.
water_route_name varchar 100 Name of the beach the water address relates to. Currently this contains the

captured segment of coastline. This will be blank for ROAD addresses.

water_name varchar 100 Water body the address relates to. This will be blank for ROAD addresses.
suburb_locality varchar 80 Dannemora Suburb/Locality from the NZ Localities (NZ Fire Service owned dataset).
town_city varchar 80 Auckland Town/City from the NZ Localities (NZ Fire Service owned dataset).
full_address_number varchar 100 1A All number components concatenated for an address.
full_road_name varchar 250 Joe Bloggs Road All road name components concatenated for an address. This has been derived

from the ‘Landonline: Roads’ Data and will move to using the new ‘Roads’ data tables when they become available’.

full_address varchar 400 1A Joe Bloggs Road, Dannemora, Auckland All address components concatenated for an address.
road_section_id integer 32 0 199943 Landonline Road Centreline ID (RCL_ID).
gd2000_xcoord numeric 12 8 174.9255518167 NZGD2000 X-coordinate for the address in metres.
gd2000_ycoord numeric 12 8 -36.9246773 NZGD2000 Y-coordinate for the address in metres.
shape geometry <geometry> Spatial geometry for the point in long/lat GD2000 ESPG 4167.
ascii variants not going to use

See Also

Process Diagram

  1. ↑ Discussed recently on the community forum
  2. ↑ 2.0 2.1 See here for information about stacked addresses
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=LINZ/Address_Import&oldid=3107418"