Proposed import of UK postcodes, UPRNs and INSPIRE IDs (England & Wales)

From OpenStreetMap Wiki
Jump to navigation Jump to search

UK postcodes, UPRNs and INSPIRE IDs (England & Wales) is an import from the Office for National Statistics UPRN Directory (ONSUD), OS Open UPRN and Land Registry INSPIRE Index Polygons datasets which is of type (data type) covering (England and Wales in United Kingdom). The import is currently (as of 2024-07-06) at the planning stage.

Goals

Add UK postcodes (addr:postcode=*) and other address tags, Unique Property Reference Numbers (ref:GB:uprn=*) (and optionally Land Registry INSPIRE IDs as ref:GB:inspireid) to buildings within England and Wales, UK.

An earlier attempt to do something similar in 2013, ONS Postcode Import was not deemed suitable for import as "The planned import could have assigned incorrect postcodes if buildings were not mapped correctly or on mixed use buildings (e.g. residential over commercial)." This preceded the release of both the Land Registry INSPIRE polygons and OS Open UPRN (2022), which are used here to prevent this issue arising. Most incorrectly aligned buildings are filtered out by use of the INSPIRE polygons and manual checking prior to upload. Using the UPRN data where a building outline contains only one UPRN avoids overwriting a correct postcode for a building with multiple postcodes.

Schedule

Proposed to be an ongoing project.

Import Data

Background

Provide links to your sources.

Data source site: https://www.ons.gov.uk/methodology/geography/geographicalproducts/nationalstatisticsaddressproducts https://www.ordnancesurvey.co.uk/products/os-open-uprn https://www.gov.uk/guidance/inspire-index-polygons-spatial-data
Data license: OGLv3 https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/
Type of license (if applicable): https://spdx.org/licenses/OGL-UK-3.0.html
Link to permission (if required): N/A
OSM attribution (if required): https://wiki.openstreetmap.org/wiki/Contributors#Office_for_National_Statistics https://wiki.openstreetmap.org/wiki/Contributors#Ordnance_Survey_OpenData https://wiki.openstreetmap.org/wiki/Contributors#Land_Registry_INSPIRE_Index_Polygons
ODbL Compliance verified: yes/no

OSM Data Files

Provide a link to the data files you have prepared for import.

Import Type

This will be a recurring import, as new buildings are always being added to OSM and larger buildings outlines split (e.g. terraces, semi-detached houses).

Data will be uploaded using JOSM.

Data Preparation

Data Reduction & Simplification

Describe your plans, if any, to reduce the amount of data you'll need to import.

Examples of this include removing information that is already contained in OSM or simplifying shapefiles.

Tagging Plans

Describe your plan for mapping source attributes to OSM tags.

Changeset Tags

Fill in the values your changesets will use.

Key Value
comment
import yes
source ONS UPRN Directory (ONSUD);OS Open UPRN;Land Registry INSPIRE Index Polygons
source:url https://www.ons.gov.uk/methodology/geography/geographicalproducts/nationalstatisticsaddressproducts https://www.ordnancesurvey.co.uk/products/os-open-uprn https://www.gov.uk/guidance/inspire-index-polygons-spatial-data
source:date (whichever is oldest of the 3 source datasets)
import:page link to this wiki page
source:license https://spdx.org/licenses/OGL-UK-3.0.html

Data Transformation

Describe the transformations you'll need to conduct, the tools you're using, and any specific configurations or code that will be used in the transformation.

Data Transformation Results

Post a link to your OSM XML files.

Data Merge Workflow

Team Approach

Initially solo, but all scripts will be made available via Github at a later date if desired by the UK mapping community.

References

List all factors that will be evaluated in the import.

Workflow

Detail the steps you'll take during the actual import.

Information to include:

https://community.openstreetmap.org/t/adding-partial-addresses-using-os-open-uprn-ons-uprn-directory-and-land-registry-inspire-polygons/115592

  • Step by step instructions
    • Find buildings in OSM which appear to be reasonably well aligned. I’m using the Land Registry Index polygons spatial data (INSPIRE) to find buildings the areas of which are at least 90% within an INSPIRE polygon. Buildings which are generally non-addressable outbuildings like garages, sheds, greenhouses, etc. are excluded. Only buildings where there is a Land Registry record a a freehold property will be found, which excludes a lot of social housing and new builds.
    • Remove duplicates, as an INSPIRE polygon which contains multiple buildings may be a site like a school where the address tags might be better placed on the enclosing amenity=school polygon (or any other number of top level tags).
    • Find the building polygons which contain only one UPRN from OS Open UPRN. This excludes buildings subdivided into flats, as they will have multiple UPRNs and there is no appropriately licensed data source to
    • Link the UPRN to the postcode from the ONS UPRN Directory (ONSUD) or National Statistics UPRN Lookup (NSUL).
    • The resulting output, with addr:postcode, ref:GB:uprn and ref:GB:inspireid tags can be conflated using JOSM. The INSPIRE ID is undocumented, but can be used to search for properties on the Land Registry website, so may be useful to data consumers (or it may be considered unnecessary bloat, in which case I’ll discard it).
    • Checking that the buildings really are well aligned and split using the the OSMUK LR Polygons layer and sanity-checking the postcodes using the CodePoint Postcodes layer can be quite time-consuming, but is absolutely necessary before anything gets into the OSM database. Usually there are still a number of buildings which still need to be realigned, split, etc.
    • Once the postcode is known, addr:city (and addr:suburb if applicable) can be added. Obviously this can be done for most addresses by just looking at one of the available map layers, but it’s helpful near the boundary between post towns.
    • If there’s more than one instance of a postcode, addr:street can be determined with reasonable confidence. Again, this can usually be done by simply looking at the map, but it helps work out to which street corner houses belong.
  • Changeset size policy - whichever is the smallest of 10 individual postcode units, 100 building polygons, or 4km^2 (the latter only applies in rural areas)
  • Revert plans - no geometry changes will be made during the conflation process, so reversion using (e.g.) the JOSM reverter plugin should be straightforward

Conflation

Downloaded OSM XML files to be processed using a Python script, applying the changeset size policy above, then loaded into JOSM for checking, adding of other address elements and upload.

QA

Add your QA plan here.

See also

The post to the community forum was sent on YYYY-MM-DD and can be found here

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Proposed_import_of_UK_postcodes,_UPRNs_and_INSPIRE_IDs_(England_%26_Wales)&oldid=2729808"