Australian Import Catalogue/City of Canning Open Data Imports

---

City of Canning Open Data Imports is an import of the city's Open Data portal, containing street features and footpaths in Western Australia.

Goals

• Add the seat and bin data as an improvement of the baseline in the area.
• Reach full sidewalk coverage with the help of the high quality sidewalk dataset.
• Potentially produce a mapping of osm <-> asset ids, where that is useful

Schedule

1. Download and Clean data (Dec 2024)
2. Evaluate data quality (Jan-Mar 2024)
3. Community post for input (Mar 2024)
4. Upload POI Changes and publish crowdsourced tracing project (Mar 2024)
5. Validate POI imports (Mar-Apr 2024)
6. Validate traced footpaths (Apr-May 2024)

Import Data

Background

Data source site: https://data.canning.wa.gov.au/
Type of license (if applicable): CC BY 4.0 with Waiver
Link to permission (if required): https://wiki.openstreetmap.org/wiki/File:OSM_CCBY_Consent-CityOfCanning.pdf
OSM attribution (if required): http://wiki.openstreetmap.org/wiki/Contributors#Australia TODO
ODbL Compliance verified: seems good to me... but hasn't been "verified" by anyone else

Data Files

• Street_Bins
• Street_Seats
• Footpath_Routable

Import Type

The sidewalk dataset has been recently completed and released. The amenity sets were existing already, and can more easily be imported.

The POIs will be imported into JOSM, visually verified against imagery, and uploaded in a single tagged changeset.

The sidewalks will be made available as a WMS/TMS tile layer for tracing, managed with a HOT Tasking Manager project.

The sidewalks may continue to receive updates, and hopefully the POIs will as well (they were created in 2015, but report that the data is as of 2025). So discovering an ongoing process of incorporating updates would be useful outcome this time. Especially as the department seem interested in making their data useful, even if they haven't found the spark yet.

Data Preparation

Tagging Plans

Operator (& Asset IDs)

The POIs are annotated with Asset_ID. I would like to find out if those IDs are stable, to potentially add them to OSM under some tag, to keep the data associated going forward.

• `operator=City of Canning`
• `operator:wikidata=Q56477868`
• `operator:type=government`
• `ref:asset_id=123` (if we decide they are useful)

If anyone finds it appropriate to add a source tag to any features added via a bulk edit, we can do that too.

Amenities

The benches are annotated as "X with Y Slats" (or "Y Batons"), indicating their material.

I would like to survey a sample of seats to see if they are consistently with/without a backrest, and if they have a consistent length, to add `backrest=` and `seats=`.

• `amenity=bench`
• `material=Y`
• `support:material=X`

The bins are annotated as "Adshell", "Canning with Advertising", "Green Bin" or "Other".

I would like to survey a sample to see if there are any other consistent properties that could be tagged.

• `amenity=waste_basket`
• `color=green` (for "Green Bin" items)

Footpaths

The footpaths are annotated with a "type", one of "CROSS", "DRIVECROSS", "FP", "IPA", "IPACROSS", "SP", "SP-NTS", as described on the dataset page and summarised here:

Footpath statistics

Type	count	type interpretation	feature descriptions
FP	2604	"footpath"	footpaths along all sorts of roads, and through parks
SP	835	"shared path"	shared cycle paths or footpaths that are a part of the City's Cycling and Walking Plan bike network.
SP-NTS	2121	"shared path - not to spec"	as above, but under 2.5m wide
(subtotal)	5560
CROSS	992	"crossing"	formal crossings protected by an island, zebra markings or signals
DRIVECROSS	93	"driveway crossing"	paths that cross driveways, mostly carpark entries
(subtotal)	1085
IPA	1504	"informal public access"	walkable verges, etc.
IPACROSS	2113	"informal public access crossing"	informal crossings with no protection

• `highway=footway`
• `footway=crossing` (for 'CROSS' and 'IPACROSS')
• `crossing=unmarked` (for 'IPACROSS')

For shared paths ('SP' and 'SP-NTS' that have dashed lines):

• `highway=cycleway`
• `foot=designated`
• `bicycle=designated`

Sidewalks are also annotated with their associated street name, which could potentially be used for some fancy validation, or importing of `sidewalk=no/separate` for `highway`s. But that's out of scope for this project.

Changeset Tags

For the two POI imports:

Key	Value
comment	Import of POIs from City of Canning Open Data Portal
import	yes
source	City of Canning Open Data Portal
source:url	https://data.canning.wa.gov.au/datasets/APPROPRIATE_LINK_FROM_ABOVE
import:page	https://wiki.openstreetmap.org/wiki/Australian_Import_Catalogue/City_of_Canning_Open_Data_Imports

The crowdsourced footpath changes will come through the Tasking Manager, and be tagged with #smart-cities-transport-project-73.

Data Transformation

The geojson files will have their properties mapped to OSM tags with the following `jq` scripts.

For Benches:

def mat:
  if . | contains("with Wooden") then "wood"
  elif . | contains("with Composite") then "composite"
  else null end;

def supmat:
  if . | startswith("Concrete") then "concrete"
  elif . | startswith("Metal") then "metal"
  else null end;

(
  .features[] | select(.properties)
).properties |= 
  {
    amenity: "bench",
    material: .Asset_Name | mat,
    "support:material": .Asset_Name | supmat,
    operator: "City of Canning",
    "operator:type": "government",
    "operator:wikidata": "Q56477868",
  }

For Bins:

def col:
  if . | contains("Green") then "green"
  else null end;

(
  .features[] | select(.properties)
).properties |= 
  {
    amenity: "waste_basket",
    colour: .Asset_Name | col,
    operator: "City of Canning",
    "operator:type": "government",
    "operator:wikidata": "Q56477868",
  }

For paths:

def highway:
  if . | startswith("SP") then "path"
  else "footway" end;

def SPdesignated:
  if . | startswith("SP") then "designated"
  else null end;

def CROSScrossing:
  if . | endswith("CROSS") then "crossing"
  else null end;

def CROSSunmarked:
  if . == "IPACROSS" then "unmarked"
  else null end;

.features |= 
  map(select(.properties?.Type != "IPA"))
|
(
  .features[]
).properties |= 
  (
    {
      highway: .Type | highway,
      foot: .Type | SPdesignated,
      bicycle: .Type | SPdesignated,
      footway: .Type | CROSScrossing,
    }
    | del(..|nulls)
  )

I selected one suburb at a time (according to the the "Location" field in the source data) to create manageable files (fish script):

for l in 'Bentley' 'Cannington' 'Canning Vale' 'East Cannington' 'Ferndale' 'Leeming' 'Lynwood' 'other' 'Parkwood' 'Queens Park' 'Riverton' 'Rossmoyne' 'Shelley' 'St James' 'Welshpool' 'Willetton' 'Wilson';
    jq '.features |= map(select(.properties.Location == "'$l'"))' Footpath_Routable.geojson | jq "$(cat format_footpaths.jq)" > footpaths_$l.geojson
end

I then loaded each suburb into JOSM, reduced the number of nodes used with the "Simplify" tool (0.15m max error), then saved it out to .osm

Data Transformation Results

Find bins.geojson, benches.geojson, footpaths.osm (all of them), and footpaths_SUBURB.osm hosted here.

Data Merge Workflow

POIs

The transformed geojson files can be opened directly as a data layer in JOSM.

The "conflate" plugin does a good job of matching existing OSM features and merging the datasets.

The locations are only accurate within a few meters, so a manual process of positioning each new entry using imagery might be needed.

Conflation statistics

Dataset	features	matches
Bins	217	5
Seats	173	0

Footpaths

Merging networks together is a notoriously difficult problem, with each dataset having features split at different points and all the rest. If anyone can do a real conflation, I'd love to hear about it! Instead, the merging process will be a manual one, broken down into chunks on a Tasking Manager.

1. The original data has been rendered to a WMS raster layer, hosted at https://maps.budgieinwa.au/canning-footpaths/{z}/{x}/{y}.png (see canning-footpaths.json and canning-footpaths.mapurl)
2. The processed footpath features have been uploaded as .osm files (per suburb for ergonomics) to allow more sophisticated tracing techniques in JOSM, such as copy-paste between layers.
3. A HOT Tasking Manager project will be created at https://tasks.smartcitiestransport.com/projects/73 to coordinate and validate manual tracing/edits to integrate the dataset.

QA

• POIs:
  • Upload the features and solicit feedback on the changeset from the community.
  • Review subsequent edits made to the features by the community to look for patterns. (TODO: BudgieInWA)
• Footpaths:
  • Verify in the Tasking Manager project, the result of the crowdsourced tracing, against Bing imagery and JOSM warnings.

See also

A post was made on the community forum on 2025-03-18 and can be found here.