WikiProject Canada/Building Canada 2020
- 1 The Idea
- 2 What is the end goal?
- 3 How to get buildings into OSM?
- 3.1 High level techniques
- 3.2 Details on importing
- 3.3 Implementation tools and workflows
- 3.3.1 Import plans: examples, tools, best practices
- 3.3.2 Getting organized: OSM Tasking Manager
- 3.3.3 Inventory of Current Building Data Sets
- 3.3.4 Creating building polygons with machine learning techniques
- 3.3.5 Community mapping projects and mapping parties: tools and best practices
- 3.3.6 Mapathons: tools and best practices
- 3.3.7 Some general tools
- 4 How to Monitor progress?
- 5 Major Obstacles to Solve
- 6 Stories, Examples and Inspiration
- 7 Events and Activities
- 8 Contacts for Groups and Organizations
Building Canada 2020 initiative (BC2020i) is an OpenStreetMap (OSM) WikiProject. It is a community-led initiative that has a vision to map in OSM all buildings in Canada by the year 2020. This vision emerged from a combination of factors: the collaborations and discussions triggered from a related (now defunct) crowdsourcing project by Statistics Canada, the growing need to improve existing georeferenced data on buildings across all communities in Canada, and a genuine opportunity to explore new forms in co-production of open data. Creating a freely accessible, non-proprietary source of nationwide information on buildings will contribute to the development of new data infrastructures and workflows of the future, upon which a multitude of public and private projects could thrive.
Simple information on buildings (e.g. WGS84-based geolocation, building footprint, full address including street # and street name, postal code, city, province and "Type Of Use") is of major societal value. This information, however, is often lacking in some areas or is not completely accessible on a single open data platform. Creating a nationwide, freely accessible and non-proprietary source of building information will contribute to development of the data infrastructures of the future, upon which a multitude of public and private projects could thrive.
This WikiProject came as an outgrowth from the pilot project done in Ottawa/Gatineau. There, the building outlines were imported from the City of Ottawa under their Open Data license. Then, the missing buildings were added and tags were added to the buildings in OSM. The value of combining Open Data to ensure completeness and accurately map the buildings with local knowledge yields data resources useful for many purposes. Although the project title says Canada the same techniques can be used anywhere in the world. In Africa, mapping buildings accurately has been used to estimate population: how many schools are needed, etc. Note the Ottawa pilot specifically made an effort to incorporate tags useful for disabled persons.
Why a community-led initiative?
Many sub-communities might be mobilized to achieve this vision; the figure at right sketches six, more might be identified. By joining OSM, we share a space where these perspectives intersect and resources are available: dialogue can occur, activities can be coordinated and goals can be achieved with efficiency in a single, open database.
As a community-led initiative, Building Canada 2020 can foster collaboration between a multitude of stakeholders (civic groups, private sector, academia, public sector...) to achieve a specific common goal benefiting each stakeholder, as well as society as a whole. This approach is inspired by principles of civic science, more specifically, civic data. A community-led approach enables stakeholders sharing a common vision to share ownership and accountability for its realization as well as the creation of its output: open, accurate and complete data on all buildings in Canada, accessible through OSM.
Launch of the initiative
A workshop was held on September 15, 2017, at Statistics Canada head office in Ottawa. Participants were invited to provide comments and suggestions about how this vision could benefit their organization or activity, the challenges they foresee, additional stakeholders that might be engaged, possible tools that could be used, possible forms of governance that could be adopted, milestones to monitor progress, etc. Ideas and perspectives from many different communities were brought together to enrich the vision and help ensure its sustainability. The workshop was attended by 52 people from 25 organizations intending to:
- Ascertain whether there was enough consensus among stakeholders to determine if it is possible and beneficial to move forward with this vision within the context of OSM.
- Outline a "high-level roadmap" that might be used to turn the vision into a reality that will benefit all.
Note that many of participants were interested in what could the data be used for rather than how to get the information into OSM.
For the Canadian economy and society, buildings are a major capital asset, the space in which a large part of economic and social activities are concentrated, an essential element of human safety and security, physical infrastructures that shape our relationship with the natural environment and contribute to energy efficiency and consumption, and are landmarks of cultural and historical identity. For these reasons, there is a constellation of information domains that relate to buildings, at municipal, provincial and federal levels.
As of today, however, there is no comprehensive, open database on buildings in Canada. Such an open database could have a multitude of uses by a multitude of stakeholders.
Governance / Project Management
This project will be managed like all other OSM projects in Canada: by the local community using this wiki, the talk-ca mailing list and the OSM Canada Task Manager to coordinate various efforts. If there are lots of volunteers and/or funding becomes available to do additional things, more elaborate coordination (a project steering committee, ongoing project management...) can be set up.
[rewrite this] This wiki (the document you are reading) helps guide BC2020's development, consensus and progress. We make progress by implementing various sub-projects and activities at a "more local level." Each of these "more local" sub-projects and activities has its own governance structure, depending on the organization or entity that implements it. Hence, Building Canada 2020 is comprised of a set of self-governing sub-projects and activities, but within the larger framework of being an OSM (Wiki)Project. (After all, OSM is the owner of the copyright of the data: in effect, all contributors who become OSM volunteers share in the copyright and ownership of Building Canada 2020's data).
Communication and coordination
Building Canada 2020 is an open and inclusive initiative. Anybody sharing its vision is invited to be part of it and connect with individuals or organizations who contribute to this vision. The following communication and coordination tools are used:
- OSM community at large: this OSM Wiki page
- OSM community in Canada: done through Talk-Ca
- Tactical how-to questions on Slack, see: https://www.osmcanada.ca/about.html (join the #buildings2020 channel)
- Tasks in a specific community: OSM Canada Task Manager (ask on talk-ca or the slack channel if you would like a new task to be created)
What is the end goal?
As no precise count of total number of buildings in Canada is known to this OSM WikiProject, we can only estimate: using the benchmark ratio of 2 to 3 people per building, the total count of buildings in Canada might range between 10 and 15 million. As of September 2017, there are approximately 2 million ways in Canada tagged building=* in OpenStreetMap. So, it is estimated that the number of buildings needing to be added to OpenStreetMap under this initiative is approximately 10 million. Rome wasn't built in a day and neither will be buildings in Canada in OSM!
The data that could be mapped
Target data types (specific tags) for this initiative might include:
- Building locations (OSM nodes) or footprints (OSM ways or relations), tagged building=* with their correct "Type Of Use" (see below),
- Address and postcode of building, using the addr=* namespace,
- Height of building in meters, tagged height=*,
- Number of floors (called "levels" in OSM), using building:levels=*, max_level=*, min_level=* and non_existent_levels=*,
- Name of building, if there is one, tagged name=*,
- Year of construction, tagged start_date=*,
- Entrance and/or exit location(s) of building, if they are known, as nodes tagged entrance=*,
- Any amenities the building may provide, as nodes tagged amenity=*,
- Other building attributes (e.g., web page links about well-known, landmark and famous buildings using the website=* tag, potentially more).
OSM key building=* currently has over 60 documented values ("Type Of Use"), including building=residential, building=commercial, technical infrastructure, whether private or public with building=industrial and/or man_made=*, educational buildings at schools, colleges and universities with a variety of values, civic and cultural buildings, etc. The man_made=* key now has over 50 documented values, some of which are not exactly buildings (adit, flagpole, snow_net, water_tap) and others which are (gasometer, observatory, silo, windmill). Amenity (Map Features#Amenity) is another useful tag. Please correctly and appropriately apply these key-value pairs as OSM documents them!
How to get buildings into OSM?
We use OSM as a data repository, so OSM tenets must be followed. If you are unfamiliar with OSM here is some starting documentation:
- Learn OSM is a good place to start to learn about OSM and is available in different languages.
- Read more on mapping techniques
- OSM user guides: Beginner, Intermediate, Advanced This one is dated, as Potlatch has been largely displaced by iD.
High level techniques
There are several methods to add buildings to OSM. And for any one area of country it is likely the various methods will need to be combined to generate a comprehensive set of building and building attributes. Some locations have existing data in other sources, some none. Some existing data have some subset of interesting attributes, others may only have the building outline. These methods require different data sources and sometimes newly-developing tools. Several approaches to entering data into OSM are discussed below.
Map it based on satellite images
A first and frequent method is by "manually" mapping building outlines. It is recommended you use JOSM and the building_Tool plugin for this. The more-beginner-friendly iD editor can be used, but it can be difficult for new mappers to accurately map buildings using iD. However, JOSM has a steeper learning curve than iD. If you are organising a group of new mappers, the OSM Canada Task Manager tiling system is recommended. It incorporates a facility for validation where someone else checks recently-entered work.
Import from Open Data sources
A second method is a formal Import of municipal and provincial data (often Open Data). As Imports are not the usual method by which data enter OSM, Imports require following OSM's Import procedures. Open Data imports need expert attention to ensure the data are high quality and meet OSM's standards for imported data. Data arising from automated image processing (e.g. Lidar data) to map buildings is currently frowned upon. This may change as potential imports move forward successfully following OSM's Import process (the steps of which should become incorporated as part of this wiki). Part of this is assuring that the licence for the data are aligned with OSM's licence (ODbL). Part is a technical alignment of the data about the buildings (which may include their height, number of floors, etc.) with OSM's data tags in a matching, harmonious, mathematical/logical mapping. (See "The data that could be mapped" above).
A third method is of greater importance and depends on local knowledge: what additional data can you add to a building which already exists in OSM? (See "The data that could be mapped" above, especially notes on the building=* and man_made=* tags). Certainly the address Map Features#Addresses, its use and other attributes. And Map Features contains other important attributes that might be added. For example, tags such as cafe and wlan are helpful to thirsty mappers wanting to check their email. There are many different (smartphone-based) apps and web-based approaches to enter into OSM these additional data, such as streetcomplete (an Android App) and Field Papers, but these involve some post-processing. Vespucci is an Android-based full editor and OSMand can be used to add POI (Point Of Interest) information. There are many other similar methods to add minor attribute data to buildings in OSM, please list your favorites here!
Details on importing
Note for properly licensed imports: A very important phase of any data importation in OSM is the logical mapping of existing data to be imported "onto" the correct OSM key-value pair tags. For example, a building dataset of an area with both a university and a city might distinguish between residential buildings which appear similar in the real world, but some are known to be dormitories (best tagged building=dormitory) while others are apartments (best tagged building=apartments). Careful consideration must preclude any effort or script to translate these data before they enter OSM. It is NOT satisfactory to simply import data in a quick, sloppy or unconsidered method hoping to then "improve them in place" with subsequent OSM edits: we must do our best to first get them right! Of course, if (building) data already exist in OSM in error, or updates are needed, such improvements are welcome.
Some building data to be imported into OSM will be sparse (perhaps only footprint and/or address data) and some will be rich (including all of the above-listed attributes and perhaps even more). The best practice is to logically map the source data onto OSM's key-value pair tags as carefully and accurately as possible. Specifying an exact list of this logical mapping from source data to target OSM tags is a critical component of any local Import Plan.
More target data types may be added to the list above. During this WikiProject's presently "open" phase, you are encouraged to add additional target data you believe might be useful. At some point, we will "close" our target data specification and no longer accept additional submissions of data types to consider. Note there is no restriction within OSM on keys or values.
Implementation tools and workflows
Import plans: examples, tools, best practices
Each data import needs to file an Import Plan with the Imports mail list within OSM.
Here are some examples, the first of which is local to this project:
- Ottawa Import Plan provides extensive documentation on how to implement a bulk import of building data from municipal open data source. This documentation includes a video tutorial, which explains the import process from A to Z (see link to video).
- Open Your City's Buildings and Addresses, and Help the Blind with OpenStreetMap is an amazing presentation from the city of Louisville (US), documenting the import process and how importing a city’s building footprint and address data into OSM can help the visually impaired with a mobile app.
- Another example from the city of Lexington KY
Getting organized: OSM Tasking Manager
WikiProject Building Canada 2020 is a massive undertaking of coordination. It benefits from organization and structure of the tasks being completed at any given time. The OSM Canada Task Manager (sometimes abbreviated TM or OSM-TM or OSM-CA-TM) is one organization tool that can be used to coordinate work.
Currently (2018-Q1), out of 42 tasks there, 33 have the word "building" in them, you can search for this to filter tasks.
Inventory of Current Building Data Sets
We are creating an inventory of existing building related datasets from municipal and provincial open data sources as well as other sources. The tables and the link below document these datasets in Canada -- openly licensed, proprietary, ODbL compatible for inclusion in OSM, or not. As some of these data source's licenses remain in a state of flux and/or negotiation with OSM's Legal Working Group (status should be indicated in the table), please pay particular attention to this before entering these data into OSM.
In addition, Canada's "Open Government" Programs website has a table of municipal open data portals that can be reviewed for building data sets: http://open.canada.ca/en/maps/open-data-canada.
Creating building polygons with machine learning techniques
This is a new and emerging area of work that has not yet been accepted by OSM due to data quality issues. It appears particularly promising with areas in Canada that are not covered by municipal open data, or to maintain OSM as "up to date." Here are some resources:
- Open source learning tools for building extraction (github codes)
- Detect missing buildings using imagery (State of the Map US 2017)
- Use Machine Learning to Create Building Heights in OSM (State of the Map US 2017)
Community mapping projects and mapping parties: tools and best practices
This section provides information on how to implement community Mapping_projects and/or Mapping_parties, aimed at adding building information to OSM. Find links to existing toolkits, best practices and lessons learned. There are examples from Canada and examples and inspiration from around the world. Some of the tools in community mapping activities include:
CODAP - The Crowdsourced Open Data Acquisition Platform - beta version
Access a beta version of CODAP here. CODAP is distributed under the MIT Licence. Code files of CODAP are available online here.
Tools and toolkits: examples and inspiration
- Community mapping for disaster risk reduction and management. Provides examples and tools on how to integrate base maps from OSM, building data, and flood hazard information to develop impact scenarios for a municipality.
Mapathons: tools and best practices
A Mapathon is a coordinated mapping event, often held indoors (sometimes known as "armchair mapping") but it can also be an outdoor or combined activity. Mapathons should OSM tutors who are happy to share their knowledge, sometimes have GPS units available to loan for the duration. If you want to run one note that it is common to offer snacks and refreshments. Using new mappers can offer challenges for data quality. If you are running one have some mice available to help the accuracy.
Building Canada 2020 can be a theme during many sorts of related events (see: Events).
Stakeholders of Building Canada 2020 can work towards the development of a set of "How to" toolkits, to help various organizations and educational institutions to plan and implement successful Mapathons across Canada.
Some general tools
Do-it-Yourself (DIY) Open Data Toolkit (Treasury Board of Canada Secretariat and OpenNorth).
An instructional manual that provides a step-by-step guide on how to develop your open data initiative. It brings together training materials, best practices, tools and resources to help you prepare for and implement an open data project.
How to Monitor progress?
This WikiProject will greatly benefit by better methods to display/demonstrate regular progress towards the goal of mapping all the buildings. It might be good to break down the status by CSD.
Active Monitoring tools
The following projects are actively monitoring the state of the buildings in Canada:
OSM Canada Task Manager is used to measure progress for many building projects in Canada: in fact, over 75% of the tasks there have "building" in their title. However, a more tightly-coupled method of the "% complete" reported there and some mechanism to "capture and report status" (on a province- or city-at-a-time basis), BC2020-wide, here or in another wiki seems a useful milestone to achieve.
[Keep adding to this list here...]
If you are looking for statistics and the location of a municipality you can search Census Profiles 2016. Here are some project-specific tables:
- Large municipalities (CSD) with population 100,000 and over. There are 54 of these. It is estimated they account for 51% of the buildings across Canada, about 7.7 million buildings.
- Medium sized municipalities (CSD) with population 50,000 to 100,000. There are 46 of these. It is estimated they account for 9% of the buildings across Canada, about 1.3 million buildings.
- Small municipalities (CSD)with population 10,000 to 50,000 There are 313 of these. It is estimated they account for 17% of the buildings across Canada, about 2.5 million buildings.
- Small municipalities (CSD) with population 1,000 to 10,000 There are 1144 of these. It is estimated they account for 12% of the buildings across Canada, about 1.8 million buildings.
The following tools can be used to generate insights on the mapping of buildings across Canada.
- OSM Cha
- R (www.r-project.org), an open Source statistical tool used for statisitcal purposes using the building tags.
Major Obstacles to Solve
The following obstacles make achieving the goal difficult.
Getting local participation
Local buy-in is needed both for support for imports and to add tags to existing buildings. Ottawa demonstrated the importance of this.
For data imports, the compatibility of municipal/provincial open data licences with OSM's licence (ODbL, Contributor Terms) needs to be verified. As it appears for documentation listed in this page, Canadian municipalities have generally used a limited set of licences types (OGL 2.0; OGL 1.0; OGL-BC 2.0, etc). What is believed to be a fairly current licence status page is under construction.
Important note: data for a number of major municipalities have already been entered into OSM; of course, their licence was cleared and accepted as compatible with OSM's ODbL. See the documentation on OSM Wiki page "Contributors".
This Building Canada 2020 OSM WikiProject is expected to facilitate a discussion of OSM's ODbL license compatibility on two fronts. First, with municipalities so that they will adopt a standard licence (a lot of work has been done on this already). (HOW? Document that here, please!) Second, the OSMF Licensing Working Group will be engaged in this discussion. (HOW? WHEN?) A necessary component of this WikiProject is to streamline a process of verification of these licences, documenting the different open data licences used by Canadian municipalities and clarify issues of compatibility with ODbL for each type of licence.
Background documentation on previous cases
The LWG has determined [(1)] that the attribution requirements of the Ottawa ODL 2.0 can be met by adding the required text to the wiki contributor page and corresponding changeset source attribute values, and that there is no downstream attribution requirement, [(2)] that we are not using "Personal Information" as defined in the licence and referenced legislation, and that so licensed material can be included in the OpenStreetMap dataset and distributed on ODbL 1.0 terms.
One of the difficulties to keep in mind is that OSM's LWG has limited resources to examine many licences: to approve "city after city after city..." is an unrealistic use of their time. A major goal of this project is to harmonize OD licenses in Canada so that as they might be used to enter building data into OSM, the LWG can be presented with a SINGLE licence (or very small number of licenses) to approve as harmonious with OSM's ODbL. This work is ongoing, progress should be documented here.
Postal Code data are not open data in Canada.
At this time, nobody is dedicated to this project, and it will only go as fast as there are people to do work on it. Organizations who stand to benefit from realizing the vision and achieving the goals of this project are encouraged to provide funding to allow people to spend more time on this project than they otherwise would. Currently (2018-Q!) the project would benefit from a Project Manager who can coordinate the many moving parts using "methods familiar to OSM." These include writing/updating this wiki to inform new and existing users and provide ongoing status as to what is going on in the project at all times, becoming a "one stop shop" to learn about and contribute to the project.
Tag Standardization / Data quality assurance
OSM building data can be assessed across the following (and more) dimensions: completeness, positional, temporal, semantic, attribute and shape/size accuracy as well as logical OSM tagging consistency. For example, some building data might include height=* or start_date=* (date built) tags, some may not, but the general OSM tagging guidelines (building tags, the Buildings wiki and the man_made=* tag) are being followed as best as possible so that data are consistent across the country. However, given the diversity that makes up Canada, 100% consistency throughout the country is a challenge. Canada-specific usage can be documented here as and when these challenges are identified.Academic institutions involved in this initiative are welcome to provide leadership on research and best practices related to QA for building data in Canada.
Existing OSM tools show some of the QA challenges related to building data in Canada. Figure 1 is an example of OSM tagging logical consistency: in this case how consistently residential building polygons are tagged with existing tags.
Data maintenance and accessibility
As with OSM in general, the ongoing maintenance to keep buildings up-to-date must be discussed. A specific plan to update data and how that will occur in the future should be outlined here.
Stories, Examples and Inspiration
In order to provide motivation for volunteers to work on this project, it is important to understand success stories of how building data (or OSM in general) is used in Canada. OSM is used in multitude of ways: on the ground by humanitarian operators, online with amazing apps and data visualization tools.... Please add more success stories here.
- The Canadian Red Cross and the Gatineau floods 2017. "During spring 2017 when Ottawa and Gatineau experienced severe flooding, the Canadian Red Cross used building information within OpenStreetMap to validate the number of homes impacted by the high-water levels. They also used these data to produce base maps of the affected area for planning and delivering assistance." (Read more ...)
- Data visualization tool for building property assessment values (Vancouver, Edmonton). "Open data facilitates innovation. I couldn’t have created this visualization without that kind of access to data." says Eugene Chen, of Darkhorse Analytics, (read the full blog post)
Other Examples and Inspiration from in Canada
- Happy Valley-Goose Bay. Here is an example of how a small municipality makes a smart use of open source technology and data (QGIS2web and OSM) to create an interactive map.
- Interactive maps created by Vancouver-based data analytics company MountainMath, show the power of combining OSM and municipal open data to visualize building and housing statistics.
- City of Edmonton, Building age map. Map created by Yourtruhome.com. in partnership with BCYEG, which found a volunteer to redeploy the Chicago Building Age map in Edmonton.
- CBC/Radio-Canada. Montréal a 375 ans, mais quel âge ont ses bâtiments? Building Age map of Montreal.
- Simple 3D buildings is a OSM wiki page that describes tags for basic 3D attributes of buildings. There is a list of amazing Demos (for example, see these F4map versions of Edmonton, Toronto and Halifax)
- Vancouver Land Prices Heat Map. Heat Map is showing price per sq.ft for Vancouver parcels of land. All of the data displayed on the map are from the Open data Vancouver Catalogue (2014 BC Assessment data from tax reports and current parcels polygons).
- Housing Price Maps (Montreal and other international cities). These maps show overall city-wide trends in residential real estate prices. The source code is also provided.
Other Examples and Inspiration from Around the world
- La Base Adresses Nationale Ouverte (BANO) est une initiative d'OpenStreetMap France. Elle a pour objet la constitution d'une base la plus complète possible de points d'adresse à l'échelle de la France. Vous trouverez sur cette page les accès aux différentes problématiques soulevées par la constitution d'une base "Adresses". In France, there are approximately 46 million building polygons mapped in OSM.
- BAG import, how the Dutch community imported nearly 9 million building footprint data and dramatically improve the quality of the Dutch address and building data.
- Future City Glasgow Maps
- Bklynr. Block by Block, Brooklyn’s past and present, with a map of the 320,000 buildings in Brooklyn, plotted and shaded according to their year of construction.
- All 9,866,539 buildings in the Netherlands, shaded according to year of construction
- Los Angeles buildings and parcels maps building and parcel size. An example of how open data are used to learn about land use in Los Angeles.
- San Francisco Building Explorer (Demo). 3D building model and information realized with the release of new building footprints to the open data portal by DataSF. (Source code available here)
- Prix immobilier partout en France Mapping of housing prices in France.
- Building Heights in England. From EMU Analytics. Emu DataPacks (Open Data on buildings) are also available for download.
- Jersey City Building Census. Read the story on Sarah Michael Levine Blog, which also include the code and explanation on "how to". And the smart city thanks Sarah. (Also available in a 3D Zoning version).
- Berlin 3D - Downloadportal. Berlin’s digital 3D city model is a joint project of the Berlin Senate Department for Economics, Energy and Public Enterprises as well as Berlin Partner for Business and Technology (read more).
Events and Activities
Please add any relevant ongoing activities and upcoming events.
- Statistics Canada Crowdsourcing Project: This project focuses on collaborating with local OSM groups and meeting with major municipalities to help them make open their data on building footprints (where available). Contact for this project: email@example.com. The following municipalities are involved:
- City of Ottawa
- Ville de Gatineau
- Spring 2018 mapathons
- July 28-30, 2018, State of the Map 2018, Milano (Italy)
- November 12-18, 2017, Mapathon Events for OSMGeoWeek at Canadian Universities, with events at Carleton University, McGill University, Selkirk College, University of Calgary, University of Waterloo, Ryerson University, University of Winnipeg, and Red River College
Contacts for Groups and Organizations
If your group or organization is or wants to be involved in this project, but not listed, please add it!
- Local Ottawa OSM group
- Toronto OpenStreetMap Enthusiasts — “Mappy Hour”
- OSMCANADA - A Community around OpenStreetMap
- Open North
- Canadian Red Cross
- McGill University - Geographic Information Centre
- Carleton University
- York University - Lassonde School of Engineering - Geomatics
- University of Calgary - Department of Geography
- Statistics Canada - Data Exploration and Integration Lab (DEIL): jean.lemullec-at-canada.ca; alessandro.alasia-at-canada.ca
- Natural Resources Canada - Centre of Mapping and Earth Observation