ONS Postcode Import
Not deemed suitable for import|
The planned import could have assigned incorrect postcodes if buildings were not mapped correctly or on mixed use buildings (e.g. residential over commercial). Although the observed error rate was low the import will not be proceeding
- 1 About
- 2 Import Plan Outline
- 3 Goals
- 4 Schedule
- 5 Import Data
- 6 Data Preparation
- 7 Data Merge Workflow
- 8 QA
This project outline is intended to help ensure that your Import Plans covers as many of the common questions about imports as possible. Just create your own page and cut and paste the editing information from this page into it.
Please! If you identify ways that this outline didn't meet the needs of your import (key evidence of this: tons of questions or alarm bells on mailing lists!), please return and fix this page.
Import Plan Outline
Link the postcode information released by the Office of National Statistics with buildings in OpenStreetMap
The intention is to run this for the first time when ONS release their February 2013 dataset, not sure whether it is worth running on a regular basis after that.
Provide links to your sources.
Data source site: http://www.ons.gov.uk/ons/guide-method/geography/products/postcode-directories/-nspp-/index.html
Data license: http://www.ons.gov.uk/ons/guide-method/geography/beginner-s-guide/licences/index.html
Type of license (if applicable): The ONS postcode directories are subject to the Open Government Licence, and Ordnance Survey OpenData Licence.
Link to permission (if required): N/A
OSM attribution (if required): N/A
ODbL Compliance verified: Yes - http://lists.openstreetmap.org/pipermail/talk-gb/2012-October/013996.html
OSM Data Files
Link to your source data files that you have prepared for the import - e.g. the .osm files you have derived from the data sources.
See Data Transformation Results
Initially it will be a one time import, although there may be value in having it as a recurring import whenever ONS release their data. It will be generated from a script. The source of the script will be referenced here when completed.
The data will be output to sets of files in osmChange format with approximately 1000 ways in each. upload.py will then be used to upload all of these using the ONS Postcode Import account
Data Reduction & Simplification
The following reduction to the data will be done for every import:
Filter the following from the ONS Postcode data:
Postcodes which have a date of termination set Postcodes whose centroid is shared with other postcodes Postcodes which have a quality indicator other than "Within the building of the matched address closest to the postcode"
Then match these filtered centroids to ways from Openstreetmap that have the following criteria:
The way is closed The postcode centroid is inside the way The way does not already have a "addr:postcode" tag The way is tagged building=*
The only tag being added is the addr:postcode tag on some ways in the UK.
I'm not planning on adding a source tag directly on the ways, the source information will be contained in the Changeset comments.
The full URL for the source file will be included in the changeset e.g. As of Feb 2013 the most up to date source of data is: http://data.statistics.gov.uk/ONSGeography/PostcodeProducts/ONSPD/ONSPD_FEB_2013_txt.zip
<tag k="created_by" v="ONS Postcode Import"/> <tag k="comment" v="Bulk import of ONS Postcode data (see http://wiki.openstreetmap.org/wiki/ONS_Postcode_Import) from http://data.statistics.gov.uk/ONSGeography/PostcodeProducts/ONSPD/ONSPD_FEB_2013_txt.zip"/>
Describe the transformations you'll need to conduct, the tools you're using, and any specific configurations or code that will be used in the transformation.
Original ONS Postcode File ▼ Filter Terminated Postcodes ▼ Filter Postcodes sharing centroid ▼ Filter Postcodes with centroid outside of a building ▼ Create Filtered ONS Postcode File
Install local version of Overpass with just UK data
for each postcode in Filtered ONS Postcode File call to local overpass to get ways (and their nodes) in area surrounding postcode centroid, Filter out ways that have no building tag Filter out ways that have a postcode tag Filter out open ways (Check if first node equals last node) Finally if a way encloses a postcode centroid put it in the output
Code for the above can be found on github: https://github.com/mcgin/osm_postcode
Data Transformation Results
Post a link to your OSM XML files.
Data Merge Workflow
This is a solo effort
List all factors that will be evaluated in the import.
Import has been discussed on Talk-GB mailing list http://lists.openstreetmap.org/pipermail/talk-gb/2013-January/014358.html
Detail the steps you'll take during the actual import.
Information to include:
- Step by step instructions: Use upload.py for each generated change file
- Changeset size policy: Approximately 1000 way's per changeset
- Revert plans: Revert changesets
Identify your approach to conflation here.
A random selection of way/postcode combinations that are identified in the output will be manually verified
Open to other suggestions.