WikiProject Canada/Canada Stats Canada Building Outlines Import/Plan

From OpenStreetMap Wiki
Jump to: navigation, search

This import is on hold pending discussion and review.

This page describes a plan to import building footprints in all major Canadian metro areas. Many buildings have already been imported in Ontario, and others are planned for other provinces/metros. The import was begun without properly following the import guidelines and without sufficient input from the OSM community. As a result, it was stopped about two weeks after it started and no new data is being imported while we decide how to proceed.

Background and Introduction

Originally a Statistics Canada endeavor to add to OSM details of non-residential buildings in Ottawa, Canada, OSM has taken initiative to import these national-scale data. It is probably best if a mapper physically "looks" at each building, but that isn't realistic with these (and such a large amount of) already-digital data; this is OSM mapping from imported data, not "on the ground" analysis of buildings and mapping from scribbled notes. The intent is not to wave a magic wand and pour all sorts of data into OSM, it is to get the best blend of these digital data and volunteer contributions of high-quality mapping skills becoming high-quality data in OSM. Canada already enjoys some (limited, but growing) success at doing this (some of these data have already been imported), so let's complete the task! One intent is to add tags to existing buildings; there are many existing buildings on the map (in Ottawa and elsewhere) that can be worked on now. Stats Canada "announced" at http://www.statcan.gc.ca/eng/crowdsourcing, but this is now firmly an OSM project, launching with this Import/Plan.

From work done with Stats Canada's building import for Ottawa, the building data are available under a Canadian Government Open Data License for other municipalities. (See Ottawa's Import wiki Canada:Ontario:Ottawa/Import/Plan). This license has been formally approved by OSM's Legal Working Group. It might not yet have made its way to the Federal Government's Open Data Portal, but that is the intention. The City of Ottawa now has 320,000 buildings of which 48,000 are tagged building=yes. Other buildings have been enriched by the local community with an average of three or four tags per building. From Ottawa, it was determined that buildings mapped using iD are not of sufficiently high quality, so to avoid future cleanup work, use JOSM instead of iD. Recent data sampling indicate the present data are quite accurate (as were Ottawa data), which bodes well for this Plan.

This project morphed from and grew into the Buildings Canada 2020 "initiative" (WikiProject_Canada/Building_Canada_2020). This suffered from a lack of direction and was partially re-written; the results of those efforts turned into this wiki. So it may be instructive to read WikiProject_Canada/Building_Canada_2020 and see if it offers additional useful historical or technical context. Some sections, especially "The data that could be mapped" may still be relevant during the present era of this project. All are welcome to copy/move sections of that (older) wiki into this one, provided they are presently relevant and achieve consensus.

It will be very ambitious to add these data to OSM, but ongoing good direction and communication should yield high-quality results. Use this wiki, its Discussion tab, the Tasking Manager and if necessary, talk-ca to communicate. Thank you for your efforts!

Permission

From Alessandro Alasia (StatCan), posted on talk-ca. Subject: "Open Building Data: an exploratory initiative":

This exploratory initiative aims at enhancing the use and harmonization of open building data from government sources for the purpose of contributing to the creation of a complete, comprehensive and open database of buildings in Canada. The outcome of this exploratory work is a first version of the Open Database of Buildings (ODB), a centralized and harmonized repository of building data made available under the Open Government License - Canada.

This initiative originates from insights taken from the Statistics Canada pilot project on data crowdsourcing, which used OpenStreetMap as a platform for integrating data on building footprints. In addition to the possible benefits of crowdsourcing, that project highlighted the potential of integrating open data from municipal, regional, and provincial governments to meet the needs of official statistics. In its current version (version 1.0), the ODB contains approximately 4.3 million building footprints. https://www.statcan.gc.ca/eng/open-building-data/index

Open Database of Buildings (ODB)

Licence

These data are published by Stats Canada under the Open Government Licence 2.0.

The OSMF Licensing Working Group determined in their meeting on 2017-03-02 that data under the Ottawa Open Data, Licence Version 2.0 (Ottawa ODL 2.0) can be included in the OpenStreetMap dataset and distributed on ODbL 1.0 terms. Quoting from the draft minutes:

The LWG has determined [(1)] that the attribution requirements of the Ottawa ODL 2.0 can be met by adding the required text to the wiki contributor page and corresponding changeset source attribute values, and that there is no downstream attribution requirement, [(2)] that we are not using "Personal Information" as defined in the licence and referenced legislation, and that so licensed material can be included in the OpenStreetMap dataset and distributed on ODbL 1.0 terms.

— OSMF LWG draft minutes, 2017-03-02 Note: The LWG's acceptance of the Ottawa ODL 2.0 or the Canada Open Data Licence 2.0 does not mean that near-identical licences are also acceptable for OSM import. For example, if the fictional City of Rotonto took the exact text of the Ottawa ODL 2.0 and merely replaced instances of “Ottawa” with “Rotonto”, the above minute indicates that the Rotonto ODL would still need LWG approval. The draft minute goes on to say:

In the past the local variants of the OGL Canada have varied widely and have in some cases included additional terms that have made them incompatible with the ODbL and in some instances non-open. For this reason we are not making a blanket statement on other such localised versions of the OGL at this point in time and will continue to review them on a case by case base.

Import Plan Outline

Schedule

An import of these data began in Ontario in late December 2018 and continued until concerns were raised about the quality of the data and the speed of the import[1]. The importers agreed to pause the import while the issues were discussed. That discussion is ongoing at the time of this writing and is now taking place mostly on the talk-ca mailing list.

Data

The data used for this import are released by Stats Canada and will be served through a data service in MBTiles Vector Tile 2.0 format, thus you must provide a z,x,y to obtain data.

Building Footprints (Polygons) https://data.osmcanada.ca/{z}/{x}/{y}/statscan-buildings.osm

OSM Data Files

The data will be served by https://data.osmcanada.ca/ as JOSM xml files via a link in the tasking manager. See data source site for more information.

Simplification

The data from StatsCan may have many duplicate nodes which will need to be cleaned up prior to import with a simplification procedure. How exactly this will be done has not yet been determined. A restart of importation should not begin until this is documented and/or completed. Preliminary analysis indicates that simplification could save several hundred megabytes in Ontario alone[2].

un-simplified building in JOSM
An example of an unsimplified building from StatsCan. 11 nodes are used to represent a rectangle.

Tags

The building footprint is the only thing being pulled in. No tags will be translated. If bad geometry is present, it will be updated, else it will be left alone. The import process can separate existing buildings in OSM vs new buildings and the main goal will be to add missing buildings as opposed to updating all geometries. Sheds will be excluded via a soft coded filter at the end of the URL(a.k.a is modifiable by importer) so that anything that is smaller than 300 sq feet is ignored (customizeable).

Attribution tags

source=Statistics Canada - Open Building Database

In changeset attribute tags will be added instead of crufting up the database with source= on each polygon

Data Workflow

Ottawa Building Import video tutorial Video explanation Video tutorial (Note: the above is NOT a workflow!)

Team Approach

Since it requires JOSM to do the import once the data have been cleaned up the actual import will be done by experienced OSM Mappers via the OSMCanada Tasking Manager. The first pass will import the buildings outlines, the second will validate the work. The hope is we can set up a team for each area:

  • coordinator
  • import data preparation
  • QA
  • import execution
  • data enrichment (commercial, residential, etc... tagging)
  • etc..

Then we can see where we have gaps and how to fill them.  Perhaps some municipalities have local mappers who have little or no technical capability to execute the actual import, though will do the tagging of building type and validation if the buildings "look right."  And maybe some folks who did imports before will help in areas where we have no technical expertise.

Risks

Risks associated with this are as follows

1. Buildings already exist in OSM


Mitigation

How we are going to avoid the problems related to the risks?

1. We will replace geometry (if really bad), using the replace geometry function in JOSM, with the new data if we determine it is better than the current. This will keep the entire history for the current item as well as keep the same OSM id. The data service can sort out existing buildings vs new ones and there will be two separate links for import. (Huh?)

2. To avoid assuming, a note should be opened on OpenStreetMap i.e. https://www.openstreetmap.org/note/741666 to request a local survey to avoid erroneous data.

QA

Quality assurance will be done by people using the Tasking Manager as well as Osmose and other QA tools to ensure proper compliance. (Needs specific steps and all tools to be used, not "just" Osmose)

Task Status

This import is on hold pending review. Please do not import any data via the links below.

Province Tasking Manager Link Completion
Ontario http://tasks.osmcanada.ca/project/145 Stopped
Québec http://tasks.osmcanada.ca/project/146 Stopped
Alberta http://tasks.osmcanada.ca/project/147 Stopped
British Columbia http://tasks.osmcanada.ca/project/148 Stopped
New Brunswick http://tasks.osmcanada.ca/project/149 Stopped
Nova Scotia http://tasks.osmcanada.ca/project/150 Stopped

References