LINZ/Howto
Contents |
A short summary of how to help with the OpenStreetMap LINZ data import
General
Getting started with JOSM and Merkaartor
There are many tutorials for the JOSM and Merkaartor editors elsewhere in this wiki, I won't try to replicate one poorly here.
The basic point is that you'll have to download one of the programs and install it locally first, and you will also need to create yourself a OSM login account to be allowed to upload data to the main OSM servers. If you don't have one of these yet, do it now:
- Go to http://www.osm.org and in the top right of the screen click on "sign up"
So far we have been using the LINZ Data Upload login account to do the bulk uploads. Ask about this on the NZOpenGIS mailing list.
If you are just starting out and a bit hesitant to mess something up, you can try uploading to the "test" server instead of the "real" one. You won't be able to view it online, but it is good for practice. In JOSM go to Edit→Preferences Connections tab set untick the default OSM server and for the OSM Server URL type:
http://api06.dev.openstreetmap.org/api
(?correct?)
Once you are confident that you are doing the right thing switch it back to use the "real" server & live database.
Getting started with the LINZ data
Using "Rob's method", described here:
The general idea is that you need to login to Rob's LINZ-2-OSM web app (ask Rob to set you up a login first)
once logged in you'll see a button on the front page:
Convert: [Run OSM Conversion]
then
Bounds: [ ] (minx,miny,maxx,maxy) Dataset: NZ Mainland V16 Layer: Coastline
The bounding box bounds are given in x for degrees longitude and y for degrees latitude. Southern and Western hemispheres get a negative sign. Thus bbox=min_x,min_y,max_x,max_y is west,south,east,north
Some example bounds for various places around NZ are given in this wiki at LINZ#Areas.
- todo: how to use OSM website/tools to create this bounding box for you. Displayed in JOSM/Merkaartor windows prior to download?
Uploading
Uploading high-resolution coastline
The idea is to replace the current PGS coastline with the much higher LINZ coastline data.
- Log in and download a chunk of LINZ coastline as an .osm file from the LINZ-2-OSM site as above. Remember uploads must be less than 50,000 nodes at a time, so don't try to upload half the South Island in one go. 20-30,000 vertices at a time is manageable. If you upload too many you'll get a ton of broken nodes which you will have to clean up manually before doing anything else.
- In JOSM or Merkaartor download the area of interest.
- Keep in mind that OSM line features have a direction to them, and that coastlines must be continuous are defined with land on the left side of the direction of travel. You might have to zoom right in to be able to see the arrow ticks in JOSM or Merk. (or switch them on manually in the preferences window).
- Do File → Open to load the new coastline layer.
- Review and explore what you see before you. Make sure your new coastline is better than existing coastline. Make sure the new data fits to what is there. Link up river mouths, beaches, rocks, etc. Compare with aerial imagery. Mix and match the data you have before you to get the best from old and new data.
- Select the old PGS coastline for removal. In JOSM do Edit → Search for the search string type :
natural=coastline source=PGS
This will select coastline features that are tagged as having come from PGS. Once the search is complete the found features will be highlighted in red. Type "3" in JOSM or F3 in Merk to zoom to them.
- Edit → Delete
- File → Upload data.
- Specify a detailed commit comment like:
Replace old PGS coastline with LINZ version: Southern Pitt Island
- It's a bit more robust to upload in smaller sections, especially if you have a flaky internet connection. At the bottom of the JOSM upload page select the "Advanced" tab, select
- [*] Upload data in chunks of objects. Chunk size: 750
- n.b.: this has nothing to do with the 50,000 feature changes per upload limit and will not help get around that.
- Finally when you are ready and have double-checked everything, click the [Upload Changes] button.
- Coastline changes are handled differently than all other layers and can take up to 3 weeks to appear. (a world-coastline shapefile is produced from the data about once a month, and the rendering is be based on that). So don't worry if you don't immediately see the change.
Uploading large polygon layers
- Keep ways smaller than 2,000 nodes and greater changesets smaller than 50,000 nodes or the 0.6 API will get upset and you'll have a big mess of orphaned nodes to clean up!
- Always run the JOSM Validator tool. The LINZ data is pretty good, but there are often some duplicate nodes and ambiguities to clean up. Aim for zero warnings and errors.
- If you want to upload just a selection, if you simply draw a box around a bunch of features it only selects the nodes and ways. You can use the Search tool to also select all "type: relation" but that also ignores any sort of spatial selection and selects them wherever they are in the entire layer. Solution: draw a box around everything you want to select, then go into the JOSM search tool and do a search for "parent selected", with the "[*] add to selection" radio button pressed. This allows a selected node to also pull in its way, and all ways to also pull in any relation they belong to.
- Relation cleanup: If you combine two outer-role ways with no inner-role present you'll get a left over empty way. When you Combine Ways you get asked to keep one and drop one outer role. Keep one, then in the multipolygon editing window click on the garbage can icon in the top of the window to drop the now unneeded relation.
- Finally, in JOSM do File → Upload selection, and confirm all selected features. See the notes above about breaking the upload into a number of smaller chunks to minimize the effect of transmission/server problems (which are common).
Cleaning
Removing duplicate nodes
Background reading:
- Go to LINZ#QA_Maps in this wiki
- "Duplicate nodes map" will show you where the dupes are.
- Zoom in on some, check to see if they are candidates for merging.
- (Things like bridges crossing streams or roads parallel to forests should not be merged (as the forest does not actually make it into the centre line GPS coordinate of the road, and (with the exception of fords) the road and the stream are not actually connected), but things like sand-ground cover transitioning into mud ground-cover or rivers connecting to lakes should be merged.)
- Open JOSM
- File → Download from OSM
- In the slippy map tab select a small area to work on and click the [Download] button at the bottom.
- After it downloads your map data should appear in the JOSM editor window.
- Edit → Search
- To search for e.g. lakes touching wetlands, search for "natural=water|wetland".
- (the '|' means "or", so add to the selection if the tag has natural=water OR natural=wetland)
- With these features now selected, click the [Validate] button in the bottom right of the JOSM window. (if it's not there try enabling the Validator plugin in the Edit → Preferences menu's Plugin controls)
- A list of errors and warnings appear in the bottom-right of the JOSM window. It is best to deal with duplicate nodes first, as that may make other errors (like overlapping areas) go away. (re-run the Validator after)
- You can open the "[+] Duplicate nodes" list and right click on one of the pairs to zoom to the problem for further inspection.
- If you are confident everything on the list should be merged, you can click the [Fix] button in the bottom right of the Validator results box to bulk fix them. Use with caution.
- Alternatively you can left-click drag a box around a node, if there are two at that point you should see two or more nodes listed in the selection box on the right side of the JOSM window. Then Tools → Merge nodes. You should see the node box change appearance very slightly.
- (use with care as it is easy to merge two nodes which are not actually duplicates, just rightly close together)
- Upload changes as above.
