Canada - The Open Database of Buildings

From OpenStreetMap Wiki
Jump to navigation Jump to search

This import is on hold pending discussion and review.

The Open Buildings Database (ODB) is a collection of open data on buildings, mainly footprints, in Canada made available under the Canadian Open Government Licence which was approved by the Licence Working Group (LWG). The ODB brings together 63 sets of data from various government sources, mainly municipalities. The purpose of this page is twofold. Provide a detailed procedure on how to import the data, and to keep an inventory of what has been done/remains to do. Organization providing the data: Statistics Canada. Format of original datasets: ESRI shapefiles.

Import Procedure

The ODB data originates from different (mostly municipal) sources and was generated in different ways. Therefore, an import has to take into consideration the specificity of the these different data sources. Hence, we cannot import all data at a national or provincial/territorial level. We have to work on a municipal basis and make sure to identify all the problems and the corrective measures to apply when dealing with issues as those identified in the Import Status section. ODB import is governed by a detailed import process that was developed by the community. Failure to comply with the procedure below would be considered vandalism and the data be deleted from the database..

Get a dedicated user account for the import

According to the Import Guideline one must create/use a dedicated user account for imports. Furthermore, by identifying the import made with the account (i.e. Source=Statistics Canada-Open Building Database in the changeset), it is not necessary to add the source tag=* on each building. However, before importing the data for a given municipality, a contributor must have first identified himself as the local import manager. Once done, the other interested users may work in collaboration with the local import manager.

This local import manager will have to identify himself by registering his dedicated import account in the proper entry of the Import status tables (in the "User" column).

Get local buy-in before importing

According to the Import Guideline, one must not import the data without local buy-in. However, and contrarily to some European country, there is might not be a local OSM community in each municipality in Canada, but only mappers sporadically adding local content from time to time. Some easy-to-use tool or Overpass queries can help identify and contact these local mappers. The local import manager must ...

  • contact them to explain their intentions by referring to the appropriate wiki pages.
  • wait a week or two for them to either answer not at all or that they have concerns, or want to help.
  • Without negative answers, the import can go ahead with the help of other contributors, if any.

Since there are only a few local OSM communities in Canada, and because Canada is large, we suggest not to limit the import of a given municipality to the people of the concerned province or region.

Ask for the Canada Tasking Manager to be set-up

The local import manager will use the Talk-ca mailing list to let the community know his intent about a given municipality. A task will be set-up in the OSMCanada Tasking Manager according to the parameters the community will have agreed on (e.g. maximum of 200 buildings in a working area). Before setting up the Tasking Manager, the local import manager may ask to have the buildings pre-processed, if necessary.

Data pre-processing includes:

  • Removal of near collinear nodes (simplification),
  • Orthogonalization of buildings (for corners having near right angles),
  • Basic tagging of available information.

Tagging: The tags produced by the pre-processing operations (p.4) are removed, except for building=yes, building:part=yes, and the addr:city tag which is copied from ODB census subdivision attribute.

Import the Data

Bulk imports are prohibited and there should be no automatic changes/conflation processes applied to existing OSM buildings. It must be done manually by carefully assessing each case! In order to ease a proper import and monitor import progress, contributors must use the OSMCanada Tasking Manager. The instructions below are valid until an ODB data server is available through the task manager. By then, you need to get the pre-processed ODB building footprints by asking Talk-ca.

Select a working area and prepare the data

If you are not familiar with the task manager operations, the documentation can be found here. The Canadian Tasking Manager’s “home page” (which lists all available tasks) displays progress of “completed” (orange bar to 100%) and “validated” (green bar covering the orange bar). A link to each approved import task is available in the Import Status section below.

  1. First, Open JOSM and make sure that the remote control option is enable and that the OSM account that will be used used by JOSM is your import acount. Then login to the Task manager by using your OSM import account.
  2. Select an available tile from the Task manager, click “Start mapping” then, “Edit with JOSM”. It will create a new layer in JOSM with current OSM content. Draw a bounding box around current tile definition (the area without yellow hatching) and copy it for the next step.
  3. Open the pre-processed ODB file. Paste the bounding box (i.e. Edit> Paste at source position). Select all ODB buildings inside the bounding box and copy them (Selection> All inside may help).
  4. Paste selected ODB buildings (Edit> Paste at source position) in the OSM layer. Once done, remove (delete) the *.buildings.geojson file from the JOSM layers, and delete the bounding box(es) from OSM-ODB data layer.
  5. Download raw GPS data over the working area (tile) from OSM (File> Download Data> Select Raw GPS data check box).
  6. Load imagery layer (ESRI World Imagery or Bing) and align it as closely as possible to ODB and GPS data (i.e. create a new image offset). Why? Because unless proven otherwise, ODB should be more accurate (XY) than most available images, especially in hilly areas.

Merge ODB and OSM Data

You should now have one layer combining OSM and ODB data, with existing OSM buildings being overlapped by the new ODB buildings.

  • Prior to proceed, you will need the “Replace geometry” tool from the “utilsplugin2” in order to keep OSM features tags and history where building conflation is required.

The actual import process starts here. In order to keep track of processed buildings, it is suggested to add a fixme tag to all the buildings before starting (Ctrl-F, search for building = *, add fixme = yes). Once a building has been checked, the fixme tag can be removed.

  1. Run the validation tool on all buildings (Ctrl-F, search for building=*; run Validation tool).
  2. Go to "Building crossing" warnings and select one record at a time. Zoom into the problem (right mouse menu) and decide if the shape and the location of the OSM building are acceptable.
    • If both are adequate, keep the OSM building and delete the ODB version.
    • Otherwise, conflate both geometries (both buildings must be selected) by applying “Replace geometry” from the “More tools” menu (Ctrl-Shift-G).
  3. Modify the geometry of the building if necessary. Align the image with ODB buildings prior to make changes. Why? Because unless proven otherwise, ODB should be more accurate (XY) than most available images, especially in hilly areas. When examining the surrounding area, add the missing buildings and remove those that do not exist anymore, if you think it is the right decision.
  4. Add/modify/delete building tags. Classify buildings with appropriate building=* tags based on imagery/local knowledge. Keep in mind that this tag is for representing the building's construction, not its current use. Here are the most common building tags you will encounter:
  5. Remove the fixme=yes tag from the building.

Clean up and upload the data

  1. Run Validation again on the whole content (not just buildings) and fix any remaining building-related warnings, such as crossing buildings/highways, fixme tags, etc.
  2. Check remaining buildings with fixme tags, if any. Remove the tag once done.
  3. Upload changes to OSM. The changesets will be tagged with source=Statistics Canada - Open Building Database.

Import Status

The following ODB inventory was built by comparing the buildings available in OSM to those offered in ODB, for each municipality. It aims at adequately monitoring import tasks by identifying the local import manager, providing a link to the corresponding task and assessing the task completion. It also compared ODB data to available Bing images to assess ODB completeness and accuracy (dx, dy).

Columns definitions

  • Municipality: The municipality that provided the data (ODB Data_prov attribute).
  • N: Number of valid buildings (OGC) found in ODB dataset.
  • User: Wiki page of the local import manager.
  • Task Manager: A link to the import task.
  • %: Proportion of the taks completed (%)
  • ODB & Bing: Particularity/comments about ODB data when compared to Bing imagery.
  • DX, DY: data offset compared to Bing’s imagery. The higher the dx,dy values, the higher the probability the OSM content will need to be fit on the imagery after aligning it to ODB data.

Alberta

Municipality N User Task Manager Link % ODB & Bing DX DY
Airdrie 22257 90% Accurate representation 3.42 -1.19
Banff 1713 80% Good representation, many large buildings are missing 1.01 -0.58
Canmore 5611 90% Good representation, some buildings need to be squared 3.2 -1.52
Chestermere 6851 90% Buildings need to be squared, many missing buildings 3.03 -1.32
Cochrane 10918 90% Multiple buildings need to be squared, many large buildings are missing -1.29 -5.12
Edmonton 320297 90% Accurate representation, many large buildings are missing 1.02 -0.67
Grande Prairie 25589 90% Missing buildings north of the city, many buildings need to be squared. 0.27 4.03
Lethbridge 49281 90% Many buildings need to be squared 18.62 1.42
Strathcona County 51038 90% Many buildings need to be squared, a few missing buildings 0.75 1.08

British Columbia

Municipality N User Task Manager Link % ODB & Bing DX DY
Chilliwack 42177 90% Many buildings need to be squared, some are missing 3.18 -0.83
Courtenay 12057 10% Good representation, some buildings are missing 1.44 -0.59
Kamloops 37478 70% Good representation, some buildings are missing 0.59 1.59
Kelowna 42070 90% Some buildings need to be squared, some are missing 12.61 -0.39
Nanaimo 27061 90% Some buildings need to be squared, large number of collinear nodes 6.03 2.65
New Westminster 8996 90% Mix of low/high quality data, some buildings need to be squared 4.29 5.08
North Vancouver 23197 50% Some low quality data (overlapping Buildings, lots, need to be squared) 1.14 -0.99
Prince George 30244 90% Good representation, some buildings are missing or need to be squared -0.89 1.07
Saanich 44964 50% Good representation, some buildings are missing -0.51 0.83
Squamish 6603 jfd553@Imports Task 173 100% Completed 16.48 3.54
Surrey 132703 50% Good representation, many buildings need to be squared 3.65 1.73
Vancouver 124130 90% Complex building representation in downtown 1.12 -0.35
Victoria 19097 10% Very detailled representation, a few missing buildings -0.09 0.98
Whistler 5266 80% Excellent overall, a few low quality data, Bing innacurate 2.35 3.85
White Rock 5123 90% Many buildings need to be squared, large number of collinear nodes, some low quality representation 3.6 3.85

New-Brunswick

Municipality N User Task Manager Link % ODB & Bing DX DY
Fredericton 25960 90% Includes sheds, many missing buildings, a few need to be squared -3.47 -1.52
Moncton 38051 90% Includes inground pools and sheds. Bing not well aligned, collinear nodes 5.23 -2.59
Saint John 31738 70% Includes sheds, some large buildings are missing -0.57 -1.56

Nova Scotia

Municipality N User Task Manager Link % ODB & Bing DX DY
Cape Breton 76588 20% Some low quality data (Small buildings have the same squared shape). Some structures provided as buildings. -0.8 -0.43
Halifax 144407 90% Good representation, missing buildings, many collinear nodes. Some structures are provided as buildings -0.1 -3.45
Nova Scotia (excluding the above) 12460 10% Only large buildings (industrial, institutions, … ) are provided. Buildings need to be squared, some low quality data. 0.08 -2.63

Ontario

Municipality N User Task Manager Link % ODB & Bing DX DY
Barrie 52117 10% Good representations, includes sheds, some missing buildings -0.32 -1.49
Brampton 130233 90% Good representations, includes sheds, some missing buildings -0.89 -0.89
Brantford 35724 10% Good representations, includes sheds, some missing buildings -0.19 -1.42
Burlington 38091 10% Good representations, includes sheds, many missing buildings 0.34 -0.22
Caledon 35005 10% Good representations, includes sheds, some missing buildings -0.38 -0.67
Cambridge 47420 10% Good representations, includes sheds, some missing buildings and some require to be squared 0.44 0.16
Durham 253690 20% Good representations, includes sheds, some missing buildings. -0.14 0.3
Guelph 39689 10% Good representations, includes sheds, some may require to be squared, many missing buildings. 0 -0.8
Hamilton 195443 10% Good representations, includes sheds, some may require to be squared, many missing buildings. -0.06 -0.93
Huron County 19550 90% Low quality buildings representation, most require to be squared, many missing buildings. -0.22 -0.68
Kingston 52705 90% Good representations, includes sheds, some may require to be squared, many missing buildings. -3.15 -1.78
Kitchener 82091 10% Good representations, includes sheds, many require to be squared, many missing buildings. 0.38 -1.39
Muskoka 88616 90% Good representations, includes sheds, many require to be squared, a few missing buildings. -1.59 -1.54
Newmarket 26784 10% Good representations, includes sheds, many require to be squared, a few missing buildings. -0.73 0.81
Niagara Region 93346 70% Good representations, includes sheds, a few require to be squared, many missing buildings. 0.86 -1.99
Norfolk County 62117 90% Variable representations, includes sheds, some need to be squared, collinear nodes, many missing buildings. -0.2 -0.4
Oakville 58180 10% Variable representations, includes sheds, some need to be squared, collinear nodes, many missing buildings. 0.15 -1.13
Ottawa 245275 99% Good representations, includes sheds, some missing buildings -0.4 -2.36
St. Catharines 52552 20% Good representations, includes sheds, some missing buildings -0.08 1.41
Toronto 418922 Nate_Wessel Task 175 70% Good representations, collinear nodes, some buildings need to be squared, some are missing -0.89 -1.04
Waterloo 32545 10% Good representations, includes sheds, some missing buildings 0.54 -2.89
Waterloo Region 34001 10% Buildings need to be squared, includes sheds, many missing buildings 0.54 -2.89
Welland 47510 90% Good representations, includes sheds, some missing buildings 0.71 -0.64
York Region 300302 30% Low quality buildings representation, differs urban/rural, many missing buildings. -0.41 0.22

Québec

Municipality N User Task Manager Link % ODB & Bing DX DY
Longueuil 63056 10% Good representations, includes a few sheds and a some buildings are missing 0.4 -1.51
Montreal 50377 60% Good representation, includes sheds, collinear nodes, many buildings need to be squared, the dataset covers 20% of the island (7 borough,very detailled - min=15cm) 0.38 -2.48
Quebec 232997 20% Good representation, some buildings need to be squared, a few missing buildings at outskirt -0.23 -2.49
Repentigny 24160 10% Good representation, buildings need to be squared, a few missing buildings 1.39 -4.75
Rimouski 29597 90% Includes sheds, above and in-ground pools, many missing buildings at outskirt -1.7 0.24
Rouyn-Noranda 33064 10% Good representation, includes sheds, old imagery 2.61 -6.55
Shawinigan 19129 90% Good representation -2.11 -2.84
Sherbrooke 457 10% ODB contains 5% of buildings (public only), Already mapped in OSM then no import are required -1.69 -6.17

Note: In Montreal, available boroughs are Ahuntsic-Cartierville, Montréal-Nord, Rosemont–La Petite-Patrie, Outremont, Côte-des-Neiges–Notre-Dame-de-Grâce, Le Sud-Ouest, Ville-Marie.

Sasketchewan

Municipality N User Task Manager Link % ODB & Bing DX DY
Regina 98329 90% Includes sheds, a few collinear nodes, missing and overlapped buildings on outskirt 0.37 1.69