notes on technical work 2010
This is archived notes dumped here by User:Acrosscanadatrails 01:55, 24 April 2010
Needs to be developed further, list of items User:Fsteggink is working on
- canvec-to-osm Python script (will be available directly from NRCan ftp server as .osm files)
- tool to split large features in shapefile, see 
- tool to merge multiple OSM files TODO
- tool to deduplicate nodes in progress (JOSM is too slow)
- part of water bodies need to be converted to coastlines manually
- Tool to split large OSM files into managable 'chunks' (or working units) Further than the NTS Grid 999xx99 into 25 smaller areas (coded "a" to "y"). (use OSM splitter)
-- User:Acrosscanadatrails 01:55, 24 April 2010
- Please note this is supposed to be a wiki talk page, not a place to keep notes. If these notes hold any meaning they should be moved to a main page, however they date back to 2010. Perhaps we could just delete this section? -- Harry Wood (talk) 09:03, 1 July 2016 (UTC)
Some old content
This is some more old content which seems to have been put here (incorrectly) by User:Acrosscanadatrails 01:55, 24 April 2010
- Import Progress
old As of a week ago, the canvec data is available to use, however, the script is still at BETA 0.49 so its gonna still change :) But my manually going through it, its easy to fix up errors.
See CanVec Import Status for the progress chart.
- Import Script
old See page canvec2osm for the current version (which is still in BETA phase) please comment on the talk-ca list, as questions will be answered much faster that way.
- Bug Reports
old Please see canvec2osm and list errors you find as another bug# or discuss on the talk page rather than using the talk-ca list (as it's likely to get lost in the archives quickly) :)
- Sample Features List
Here's the chart for the CanVec OSM Map Features showing the sample features uploaded list.
Older Status of CanVec Map Features Upload Samples
(The the wiki-charts might be out-dated and should be removed) --acrosscanadatrails 04:05, 20 November 2009 (UTC)
old, chart missing, is in Google Docs Included in the chart below, are 2 extra columns. Please sign your name on the notes, and explain why you approved/disproved with it. Also indicate indicate approved or disapproved in the column.
The purpose of voting AND explaining your position, is to make this import the 'most right'. If you have further comments, please use the talk page. Once similar answered are revielved, we can break it down to defult answers for others to choose from.
CanVec - Feature Catalogue, Edition 1.0.2 (PDF version, 1,123 KB) This document describes all of the CanVec's features and their attributes. More ifnormation can be found at the [Canvec/OSM Map Features|OSM Map Features] page.
Garmin IMG tiles
I have converted 4 of the 6 sample canvec.osm.zip files into a routable Garmin IMG tile.
All of the files (for all data) is available in the "NTS Grid (all data)" folder http://www.mediafire.com/acrosscanadatrails
-- User:Acrosscanadatrails 01:55, 24 April 2010
- Please note this is supposed to be a wiki talk page, not a place to keep notes. If these notes hold any meaning they should be moved to a main page, however they date back to 2010. Perhaps we can just blow away this old content -- Harry Wood (talk) 09:03, 1 July 2016 (UTC)
too much wood?
The natural=wood areas in Canada look weird. Take for example this map. The first thing that strikes me is that there are *squares* of forest, which is obviously an abberation. The second thing of course is that some forest is missing. I would be quite surprised if the forest stops abrubtly like that north of Ottawa. The third thing is that there are lines crossing the forest which also seem to be an abberation. I assume that is related to the way the data is formatted, but it should be fixed.
Lastly and more importantly: the data makes national parks and such important features mostly impossible to distinguish (if they exist at all). The comparison with US data (vermont, in this case) is striking.
Oh and another thing, not sure it's related: there's this weird triangle here, and only at that zoom level...
-- TheAnarcat 22:42, 4 January 2012 (UTC)
- I find this to be a serious problem as well. The good news is that it seems that the layer containing this data is separate from the data for things like parks, at least it seems that way when I watch the image draw up in the JS editor. Perhaps the results would be improved by simply not importing this data? Maury Markowitz 13:28, 16 October 2012 (BST)
- Yes zoom level 8 does look like a total dog's dinner across much of Canada because of the very messy squared off areas of forest coverage. That's my impression every time I look at the map of Canada, but I was noticing it recently because someone was suggesting this image as one of the Featured image proposals
- Potentially a nice topical news story for OSM, but we don't want to feature an image which shows such messy inconsistent forest data. This case also begs the question, how would anyone sell the idea of using OpenStreetMap during a wildfire emergency response (or indeed for any other purpose), when the map of forested areas (before wildfire) bares so little relation to reality?
- Is that how the forest data is within the canvec data? or just how it was imported? What was the plan for data clean up after this import?
- -- Harry Wood (talk) 12:11, 1 July 2016 (UTC)
- Harry — there's some ‘spirited discussion’ going on about this right now on the talk-ca list after a mapper from away decided he didn't like the way Canada looked and started deleting stuff. To your questions: 1) This is how forests appear within CanVec, though some users make an effort to clean it up on import. 2) Some of the data are very old, and the forest and lake surveys don't match in age. You might have forest surveys from the 1970s mixed in with lake surveys twenty years newer. The boundaries often don't quite match. In the CanVec metadata, you can see these dates, along with an accuracy estimate (often not very; I've seen 150m applied as standard tolerance over large areas). 3) Plan for data cleanup? We'd like to have one, but there are so few mappers in Canada, and even fewer with the kind of computational power (or time, or interest) needed to clean up multi-thousand km² forest polygons each with a few thousand lakes, that we may never get there. --Scruss (talk) 12:56, 2 September 2016 (UTC)
Algorithm to determine NTS grid designator from latitude and longitude
Fixing continuous vectors
Several helpful people have pointed me to this page, so here goes...
I have noticed that the CanVec data is split up across tiles, as described on the associated page. Unfortunately this causes problems for any features that cross the grids, as they get split up into multiple unrelated parts. I first noticed this for Mackenzie Lake, and when I expanded my view I noticed this through the entire map.
It appears that "fixing" this is a manual process of reconnecting dots. Is this really the only solution? Does the CanVec database not include some sort of ID number that we could use to automate the re-linking?
Maury Markowitz 16:57, 16 October 2012 (BST)
- Yes, this is a huge problem with the existing imports, and given that the map ‘looks okay’, may never be fixed. Unless we can find a team of incredibly picky people with near infinite computing power and lots of free time … --Scruss (talk) 23:09, 15 November 2014 (UTC)
- It looks less okay for these natural=glacier relations, just because in this standard style, these are rendered with an edge on the polygon, so the straight lines edges show up pretty prominently.
- I'm not sure if joining together the squares is the best thing to do anyway, because they have quite a lot of intricate complexity (too much detail for any normal mapper to be maintaining actually), and so the multi-polygons will start to use a very unwieldy amount of data if they were all joined together.
- The best solution (given lots of time of volunteer fixup mappers, or a very clever automated algorithm) is to join them together to eliminate the straight lines, but then find ....more logical ...places to chop up the polygons e.g. following some river course or other natural division between the chunks.
- There's some examples of this in northern india actually. Some really crappy slabs of woodland (VMAP0 data I think) were dumped on the map. From the import, they are split into irregular polygon chunks (not squares, but still long crude straight lines representing nothing in reality). In many places that's still very visible. In some places mappers have worked, firstly to add much more detail to the data, but also to rationalise by re-dividing the chunks of woodland along valley bottoms.
- It's a huge post-import fixup challenge you have in Canada!
- -- Harry Wood (talk) 14:41, 29 September 2016 (UTC)
Too much detail!?
I'm the only one who thinks that the CanVec import sometimes contains too much details? I mean some highways have a node every 10-12 meters even if it's straight. E.g Countryview Drive http://www.openstreetmap.org/?mlat=44.70125&mlon=-63.56487#map=19/44.70125/-63.56487
- Absolutely. There is no reason to have an untagged node in the middle of a straight line. —Michael Z. 2014-09-16 22:19 z
CanVec+ vs CanVec
Seems that CanVec+, and the updated product is renamed back to CanVec … wha? Source here: http://open.canada.ca/data/en/dataset/e2c70a40-e4dc-11e3-a555-90b11c67bbbf -
“This collection is a legacy product that is no longer supported. Users of CanVec+ (release of October 29, 2015) should plan to make the transition towards the new CanVec product.”