Import/Catalogue/PSMA Admin Boundaries

From OpenStreetMap Wiki
Jump to: navigation, search


PSMA Admin Boundaries is an import of the PSMA Suburbs-Localities and Local Government Areas datasets which are administrative boundaries covering Australia. The import is currently (as at 2018-10-16) at the planning stage.

Goals

The goal is to bring increase the coverage of Suburbs, Localities and LGA boundaries in Australia to close to 100%. Prior to the import many of these features are either not in OSM or only exist as points.

The import has been extensively discussed within the local community and the community is pro-import, https://lists.openstreetmap.org/pipermail/talk-au/2018-October/thread.html

Schedule

In planning/discussion: Q4 2018

Import Data

Background

Data source site: https://data.gov.au/dataset/psma-administrative-boundaries
Data license: https://creativecommons.org/licenses/by/4.0/
Type of license (if applicable): CC BY 4.0
Link to permission (if required): File:Department_of_Industry_Innovation_and_Science_ODbl_permission_Administrative_Boundaries.pdf
OSM attribution (if required): https://www.openstreetmap.org/copyright
ODbL Compliance verified: yes

OSM Data Files

(see Data Preparation for documentation on how these were created)

We don't yet have a consensus if we should import the full geometries or simplified geometries.

Import Type

One-time import, with potential of followup semi-manual imports to account for changes in each quarterly upstream release.

Identify what method will be used for entering the imported data into the OSM database - e.g. API, JOSM, upload.py, etc.

Data Preparation

Data Reduction & Simplification

Before the import we should remove existing administrative=boundary and place=* ways or relations IF the place exists in the data we are importing.

Tagging Plans

LGA

Left empty for mappers to manually add in later:

Suburbs/Localities

Left empty for mappers to manually add in later:

Changeset Tags

Data Transformation

https://github.com/andrewharvey/psma-admin-bdy2osm documents the work in progress approach to get the data into shape for an OSM import. The transformations it makes are summarised as:

  • Download upstream data
  • Convert into per state GeoJSON files with OGR2OGR
    • For Suburbs/Localities only "GAZETTED LOCALITY" and "DISTRICT" are extracted, specifically "HUNDRED"s in South Australia are omitted.
  • Transform into OSM schema GeoJSON files with custom transform code
  • Convert GeoJSON to TopoJSON to detect topology and shared ways between LGA and Suburb/Localities
  • Convert that TopoJSON into OSM XML relations using topo2osm

Data Transformation Results

Data Merge Workflow

Team Approach

To be determined, lots of interested people in the community, need to coordinate.

Here is a very work in progress chart of who will be doing what:

Task Person
Import Account/Upload Andrew Harvey
QLD Integration Joel Hansen
SA Integration ...
WA Integration ...
TAS Integration Lee Mason
NT Integration ...
OT Integration ...

References

List all factors that will be evaluated in the import.

Workflow

Detail the steps you'll take during the actual import.

Import by State
LGA Suburb/Localities
1. NSW Done. Comparison of existing boundaries here Complete, major differences are due to the use of pre-existing rivers or coastline. Smaller differences could be manually edited if needed.

Comparison of existing boundaries here

2. VIC most existing, no import most existing, no import

however many differences,

need local knowledge

3. QLD has some, to be replaced none existing, to be imported
4. SA many existing, partial import Complete, some need manual editing

Comparison of existing boundaries here

5. WA 3 existing, to be replaced none existing, to be imported
6. TAS 0 existing, to be imported none existing, to be imported
7. NT 1 existing, to be replaced none existing, to be imported
8. ACT N/A
9. OT 0 existing, to be imported none existing, to be imported

Select all ways and add boundary=administrative and source=PSMA_Admin_Boundaries tags.

Drop water based unincorporated LGAs.

Information to include:

  • Step by step instructions
  • Changeset size policy
  • Revert plans

Work in Progress:

  • Choose a state, as the import will be done one state at a time
  • Use existing state boundary
  • Apply any coastline conflation if using the existing coastline as some of the boundaries (or this can be done post-import)
  • Currently, the upper limit for the size of a changeset is 10,000 objects, however JOSM can workaround this by automatically spreading an upload across multiple changesets automatically https://josm.openstreetmap.de/wiki/Help/Action/Upload#Runningaverylargeupload
  • Done using a dedicated imports account

Conflation

Most if not all localities in Australia are already mapped as nodes. We should take this into account as we blend the PSMA data with OSM data.

  1. Add a place=* node if one is missing for the locality/LGA.
  2. Make sure the boundaries have a place=* tag, have it match what is on the nodes.
  3. General spell and capitalisation check on names
  4. Add the nodes into the relation, and set the role to "label" (example: https://www.openstreetmap.org/relation/6070036)

And for general improvement to OSM, the following are highly recommended

  • Reconsider the place=* value (Is it really a village or a suburb? Is it really a city or is it a town?)
  • Reconsider the placement of the node (For rural places it should be roughly in the centre, for populated areas put it near a central town hall, square or similar.

QA

Add your QA plan here.

See also

The email to the Imports mailing list was sent on 2018-10-22 and can be found in the archives of the mailing list at https://lists.openstreetmap.org/pipermail/imports/2018-October/005760.html

The discussion within the Australian community was ongoing on the talk-au mailing list at https://lists.openstreetmap.org/pipermail/talk-au/2018-August/thread.html and https://lists.openstreetmap.org/pipermail/talk-au/2018-October/thread.html