Import/Catalogue/The Department of Environment and Resource Management/Peaks And Mountains Import
STATUS
Manual import of parcels of data underway - estimated completion in 2 or 3 weeks. Chas66 (talk) 06:31, 9 May 2014 (UTC)
This page documents a proposed data import: peaks and mountains in Queensland Australia from State Government Place names gazetteer Data Shapefile.
Source Dataset:
- Title: Place names gazetteer - Queensland
- Alternative title: IQATLAS.QLD_PLACE_NAMES_GAZETTEER
- Date: 2014-04-20 (publication)
- URL/service: http://dds.information.qld.gov.au/dds?title=Placenamesgazetteer
- Licence:Creative Commons - Attribution 3.0 Australia licence.
- Required import Attribution:
? State of Queensland (Department of Natural Resources and Mines) 2013. Updated data available at http://dds.information.qld.gov.au/dds/
Proposed Import Steps
- Refine shapefile dataset to import
- Extract the Rows containing Mountains and peaks names:
- Shapefile column: TYPE == "MT" or "PEAK"
- CSV version of this data is here: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/ShapeFileExtractQldPeaks2014.csv
- Extract the Rows containing Mountains and peaks names:
- Find and remove obvious duplicates (same location with same or different name)
- Have used OpenRefine to clean up the data http://openrefine.org
- Example true duplicate rows removed:
- Mount Elphinstone -27.503967, 152.896702
- Mount Elphinstone -27.503967, 152.896702
- Different name, but same location:
- This mean it is a Mountain(Feature Extent) and a Peak with diff names, or,
- Another way of storing an ALT_NAME
- For our purposes we can store as an ALT_NAME - action taken was to remove the duplicate row, and add it's PLACE_NAME as ALT_NAME to the first row found at that location. Example:
- Arthurs Mount -23.05, 142.9
- Walters Knob -23.05, 142.9
- made Walters Knob ALT_NAME for Arthurs Mount and removed Walters Knob row
- Exception is these twin peaks peaks, one of which must just have the wrong location I believe:
- Mount Harriet -23.298611, 138.167786 http://www.nrm.qld.gov.au/property/placenames/details.php?id=15429
- Mount Kate -23.298611, 138.167786 http://www.nrm.qld.gov.au/property/placenames/details.php?id=17794
- Update : NRM Qld Govt have offered corrected data for these two, they are really ~390m apart at these locations:
- Mount Kate 23d17m26s S 138d10m18s E
- Mount Harriet 23d17m37s S 138d10m10s E
- refined/pruned data has 2563 nodes
- converted to OSM format file with following tags using ogr2osm python utility.
- OGR2OSM python source translation file is here: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/translationQldPeaksDraft.py
- The output of this, an OSM file (the one to start manual review with) is here: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/QldPeaks.osm
- At upload time these Changeset tags to be added manually (to the changeset):
source = 'Place names gazetteer - Queensland Dated: 2014-04-20' source_ref = 'http://data.qld.gov.au' attribution = 'State of Queensland (Department of Natural Resources and Mines) 2013. Updated data available at http://dds.information.qld.gov.au/dds/'
Manual Review Import Process
- Get existing nodes tagged with natural=peak in the database using the overpass api
- bounded by Queensland area box, get JOSM file of all natural=peak nodes
- http://overpass-turbo.eu
- See this file for the query used: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/overpassQuery-PeaksInQld.txt
- This file is here: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/OSMextractedQldPeaks.osm
- shows 569 relevant POIs exist in the database
- Propose import to community
- manual cross reference to existing 569 POIs CURRENT STAGE
- If node Duplicated, prefer existing location
- assume existing data quality is better than this dataset
- However, if spelling differs, prefer Gazetteer version
- If node Duplicated, prefer existing location
- manually updated JOSM import file
- upload with new User ID "QLD_Peaks"
Elevation Data
Elevation could be added, by using the osmosis tool with the SRTM plugin at: https://code.google.com/p/osmosis-srtm-plugin
This plugin downloads the NASA 2.1 SRTM data for the nodes, and adds a tag with the height. The tag should be ele= , and the height in metres doesn't need to be a float, so cleanup will be required (rounding to nearest whole metre should be sufficient).
The ogr2osm tool doesn't create a valid 0.6 OSM file according to OSMOSIS, it needs a version and timestamp attribute on the node. Amending the input file was required by search/replace the following in the input OSM XML to get it to process in OSMOSIS:
s/g/<node /<node version="1" timestamp="2014-05-04T06:37:45Z" /
command used to add the heights:
~/src/osmosis-latest/bin/osmosis --read-xml ./QldPeaks.osm --write-srtm tagName=ele --write-xml test_with_srtm.osm
A data file containing the output of that command; with the elevation tag and value added, is here: https://dl.dropboxusercontent.com/u/446994/osmQldPeaks/test_with_srtm.osm
UPDATE: whilst adding height from SRTM is possible, discussion on the mailing list has leaned toward not adding the elevation/height from the SRTM. This elevation tag would not be imported
Sample tags of imported Node
<node id="-634" version="1" timestamp="2014-05-04T06:37:45Z" lat="-18.35" lon="144.0166667"> <tag k="name" v="Beril Peak"/> <tag k="natural" v="peak"/> <tag k="alt_name" v="Blackman Peak"/> </node>
Manually add these tags to the changeset at upload time:
source = 'Place names gazetteer - Queensland Dated: 2014-04-20' source_ref = 'http://data.qld.gov.au' attribution = 'State of Queensland (Department of Natural Resources and Mines) 2013. Updated data available at http://dds.information.qld.gov.au/dds/'
Sanity check of imported data
Adding the GA WMS service as an image layer of their NATMAPS 1:250000 raster maps and browsing the mountains on the WMS image layer to JOSM with the OSM file loaded allows coarse check of alignment (the GA maps are built from the same data). This check ensures nothing untoward has skewed the data points position.
WMS server end point: http://www.ga.gov.au/gisimg/services/topography/NATMAP_Digital_Maps_250K_2008Edition/ImageServer/WMSServer?request=GetCapabilities&service=WMS