Import WRI Congo Basin Forestry Roads
Goals
The World Resources Institute has released forestry road data for a set of 6 countries in Central Africa to be imported into OSM. The involved countries are: Cameroon, Equatorial Guinea, Central African Republic, Gabon, Democratic Republic of Congo and Republic of Congo.
The goal is to manually import all the road segments not yet present in OSM, joining them with the existing highways.
For this purpose, a detailed workflow will be followed by all users involved in this task.
Schedule
- Preparation, discussion - due to start February 9th 2015.
- Import - expected to start any time after the community has solved any issues, doubts or concerns about this import.
Import Data
Data description
The original data is in shapefile format. The accuracy of the data varies: some segments are quite correct and can be imported with little correction, whereas some others have to be corrected manually before upload or being used as a guide to map the actual road.
Methodologies for production process of source data was the same across all congo basin countries. Detailed on the origin and production process for the source data can be found here.
Original tags vary for each country, but are quite similar. The proposed OSM tagging tries to keep consistency across all the countries.
Original files have the following number of objects:
| Country | Number of nodes | Number of ways |
|---|---|---|
| Cameroon | 131,747 | 8,551 |
| Equatorial Guinea | 10,185 | 239 |
| Central African Republic | 16,803 | 1,823 |
| Gabon | 445,015 | 10,378 |
| Democratic Republic of Congo | 2,630 | 973 |
| Republic of Congo | 41,302 | 1,764 |
| TOTAL | 647,682 | 23,728 |
Background
ODbL Compliance verified: YES
WRI has given full authorization for the use of their data with the standard authorization document of the Humanitarian OpenStreetMap Team (HOT). A scan of the document can be found here.
Import Type
As the quality of the data varies, some ways can be imported with few corrections, but others will rather be used as a guide for tracing a more accurate one, transferring the tags from the WRI segment to the new one. Incorrect segments or segments will be ignored (deleted), whereas segments that overlap with OSM data will be conflated.
The import will be carried out by users who will read and understand the workflow wiki set for this import. For each of the 6 countries, the process will be done with the help of a HOT Tasking Manager project, where every mapper will download the OSM data and the WRI data into two different layers, so they can correct the WRI data and conflate it with the OSM data before they upload it to the OSM database.
Data Preparation
Data Reduction & Simplification
The data is originally in shapefile format, in only one file for each of the 6 countries. You can download the original files here.
From this files, we extracted the forestry relevant data. We further deleted all segments that have been already mapped in OSM.
Data has been corrected of errors, duplications, overlaps, etc., but it will be checked again during the import process.
Tagging Plans Country by Country
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| Shape_Leng=* | Not tagged because not relevant | |
| code=* | Not tagged because not relevant | |
| indicatif=* | Not tagged because not relevant | |
| last_edi_1=* | Not tagged because not relevant | |
| last_edite=* | Not tagged because not relevant | |
| n_origin=* | start_date=before YYYY-MM-DD | Original values are like YYYYMMDD |
| n_yrXXXX=* (XXXX=9900, 0001, 0102,..., 0910) | All are checked together to see if abandoned, and if yes, abandoned:date=* with earliest year it is abandoned. | Values are Abandonned road, Active road, Cloudy, No data and Non active road |
| objectid=* | Not tagged because not relevant | |
| prioritair=* | Not tagged because not relevant | |
| roadtype=Primary logging road | highway=track + access=forestry | If road is abandoned, then abandoned:highway=track instead of highway=track |
| roadtype=Secondary logging road | highway=track + access=forestry | If road is abandoned, then abandoned:highway=track instead of highway=track |
| roadtype=Forest road | highway=track + access=forestry | If road is abandoned, then abandoned:highway=track instead of highway=track |
| roadtype=Plantation road | highway=track + access=agricultural | If road is abandoned, then abandoned:highway=track instead of highway=track |
| shape_STLe=* | Not tagged because not relevant | |
| type_route=* | Not tagged because not relevant | |
| ALL | source=Landsat;WRI |
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| Toponimia=na | Not tagged because not relevant | |
| Categoría=Pista forestal | See dscripció=* further down | |
| Cod_WRI=* | start_date=before YYYY | 4 values: if 0.0 not imported, if 2001.0 YYYY=2002, if 2002.0 YYYY=2003 and if 2003.0 YYYY=2004 |
| Comentario=na | Not tagged because not relevant | |
| dscripció=* | All values, with Categoría=Pista forestal, give us access=forestry | 5 values are Carretera privada principal, Carretera privada secundaria, Carretera privada terciaria, Carretera pública terciaria and Carretera secundaria |
| Estado2012=* | Not tagged because not relevant | |
| Fecha_act=* | source:date=YYYY | 2 values: 2012 and 2013 |
| Long_km=* | Not tagged because not relevant | |
| Orig_datos=* | Not tagged because not relevant | |
| Sce_geo=* | source:geometry=INDEFOR-AP cartografía | |
| Shape_Leng=* | Not tagged because not relevant | |
| Tipo_carrt=No asfaltada | surface=unpaved | |
| Usage=* | If No activa, abandoned:highway=track, else highway=track | |
| ALL | source=Landsat;WRI |
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| SHAPE_STLe=* | Not tagged because not relevant | |
| Shape_Leng=* | Not tagged because not relevant | |
| annee_ferm=* | abandoned:date=before YYYY | 4 values: if 0 not imported, if 2009 YYYY=2010, if 2010 YYYY=2011 and if 2012 YYYY=2013 |
| annee_ouv=* | start_date=before YYYY | 6 values: if 0 not imported, if 2000 YYYY=2001, if 2008 YYYY=2009, if 2009 YYYY=2010, if 2010 YYYY=2011 and if 2011 YYYY=2012 |
| created_da=* | Not tagged because not relevant | |
| created_us=* | Not tagged because not relevant | |
| last_edi_1=* | Not tagged because not relevant | |
| last_edite=* | Not tagged because not relevant | |
| long_km=* | Not tagged because not relevant | |
| source=* | Not tagged because not relevant | |
| statu_rou=* | If statu_rou=ouvert => highway=track. If statu_rou=ferme => abandoned:highway=track | If this tag is not present, then highway=track |
| type_route=* | 5 values: if 7, 8, 9 or 10, then access=forestry. If 11, access=agricultural. | |
| type_surf=terre | surface=unpaved | |
| ALL | source=Landsat;WRI |
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| Annee=* | start_date=before YYYY | 11 values: if 0.0 or 10431.0 or 206.0 not imported, if 2004.0 YYYY=2005, if 2005.0 YYYY=2006, if 2006.0 YYYY=2007, if 2007.0 YYYY=2008, if 2008.0 YYYY=2009, if 2011.0 YYYY=2012, if 2012.0 YYYY=2013, and if 2013.0 YYYY=2014 |
| CODE=* | Not tagged because not relevant | 7 values: 0, R1, R2, R3, R4, R5 and RES/Rt |
| CODE_BD=* | highway=track + access=forestry. In case it's abandoned according to Etat_YYYY=* tags: abandoned:highway=track + access=forestry. If together with Nature=* it will be tagged according to that tag (see Nature=* further down) | 1 value only: ROUTE D'EXPLOITATION |
| Etat_YYYY=* (YYYY=2004, 2005, 2006, 2007 and 2008) | All are checked together to see if abandoned, and if yes, abandoned:date=* with earliest year it is abandoned. | Values are Abandonné (or similar with typos), Fréquenté (or similar with typos) and, for Etat_2007=*, there is a third value: No data |
| Id=* | Not tagged because not relevant | Same as Annee=*, but without the .0 at the end. |
| Long_km=* | Not tagged because not relevant | |
| Nature=Aviation | aeroway=runway or aeroway=taxiway | As they total 8 segments, they will be tagged manually as either runway or taxiway |
| Nature=Gares | railway=rail + {{ta<different>g|service|spur}} | |
| Nature=Mines | highway=track + access=private | |
| Nature=Plantation | highway=track + access=agricultural | |
| Shape_Leng=* | Not tagged because not relevant | |
| Type_Image=* | Not tagged because not relevant | |
| ALL | source=Landsat;WRI | |
| All segments without highway=track nor abandoned:highway=track nor Nature=* | highway=track |
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| Date_orig=* | Not tagged because not relevant | Date of reference satellite image |
| Shape_Leng=* | Not tagged because not relevant | |
| categorie=Forestière | highway=track + access=forestry | |
| categorie=Chemin de Plantation | highway=track + access=agricultural | |
| categorie=* | If this tag is not present: highway=track | |
| code_wri=* | Not tagged because not relevant | |
| dat_maj=* | Not tagged because not relevant | same as Date_orig=* |
| dat_obs=* | start_date=before YYYY-MM-DD | Date road was first digitized |
| long_km=* | Not tagged because not relevant | |
| long_m=* | Not tagged because not relevant | |
| num_ccf=* | Not tagged because not relevant | |
| num_ga=* | Not tagged because not relevant | Code for logging concession road falls into |
| org_donnee=* | Not tagged because not relevant | |
| sce_geo=* | source=WRI;DIAF;Africover or source=WRI;DIAF;Société Trans M or source=WRI;DIAF, depending on the value | 3 values: Africover, Société Trans M and DIAF. |
| sce_sem=* | Not tagged because not relevant | Organization which compiled/ complete data: DIAF. |
| type=* | Not tagged because not relevant | 4 values: Exploitation Sec, Futur Princ, Futur Sec, Public |
| WRI Tag | OSM Tag | Comments |
|---|---|---|
| Annee=* | start_date=before YYYY | 8 values: 0, 1988, 2000, 2001, 2002, 2003, 2006 and 2007. If 0 not imported. For the rest of the values: XXXX -> YYYY=XXXX+1 |
| CDE_VCOM=YYYY.0 | Not tagged because not relevant | |
| DESCRIPTIO=Route forestière principale | access=forestry | |
| DESCRIPTIO=Route forestière secondaire | access=forestry | |
| DESCRIPTIO=Route d'exploitation | access=forestry | |
| DESCRIPTIO=Route agro industrie | access=agricultural | |
| ETAT=* | If ETAT=Abandonnée, then abandoned:highway=track. Else, highway=track | 2 values: Abandonnée and En service |
| Long_km=* | Not tagged because not relevant | |
| Longueur=* | Not tagged because not relevant | |
| Periode=* | When together with source:geometry=Aster and not year 1899, then source:date=YYYY-MM-DD | Periode is the source date of Aster aerial imagery |
| SOURCE=* | source:geometry=* . 3 values: source:geometry=Aster, source:geometry=DMCii and source:geometry=LandSat | |
| Shape_Leng=* | Not tagged because not relevant | |
| Usage=* | Not tagged because not relevant | 3 values: Nulle or nulle, Permanente and Temporaire. |
| ALL | source=WRI |
Changeset Tags
We will use the following changeset tags:
- comment=WRI Forestry Roads import. NAME_OF_COUNTRY #project-NUMBEROFPROJECT
- source=WRI;Landsat;imageryProvider
- import=yes
- url=https://wiki.openstreetmap.org/wiki/Import_WRI_Congo_Basin_Forestry_Roads
where imageryProvider=Bing, Mapbox or whatever imagery used while importing the data, NUMBEROFPROJECT being the Tasking Manager project number and NAME_OF_COUTRY be substituted by the country code: CMR for Cameroon, GNQ for Equatorial Guinea, etc.
Data Transformation
Each of the 6 files in shapefile format were opened with JOSM + Open Data plugin, and saved in osm format. After this, each one was transformed into a new osm file with the proposed tags, using an ad-hoc awk script for each one.
You can download those final files and scripts here.
Data Merge Workflow
Team Approach
Import will be undertaken by mappers trained for this task, using an import specific OSM user account and following a [WORKFLOW LINK detailed workflow with screenshots], that shows all the steps to follow, with the goal of best keeping the consistency and correctness during all the process.
References
Following the imports guidelines, this import will be discussed first in the local mailing list for R. D. of Congo, Cameroon and Central African Rep., and in the HOT mailing list for Gabon, Equatorial Guinea and Rep. of Congo, as they don't have a local OSM chapter.
With the inputs from those lists, we will open a thread in the imports list.
Workflow
The full workflow wiki can be checked here.
This workflow will deal with the following issues:
1. In areathat will be s with high resolution aerial imagery, users will check the accuracy of each segment against that imagery, and will correct it or trace a new one using the WRI segment as a guide if accuracy is poor.
2. Conflating with the existing OSM highways: As explained before, the roads in the WRI dataset that conflict with others already in OSM have been removed already. If one user finds a segment conflicting with an OSM highway, he/she will check which one is more accurate. In case the OSM way is better, s/he will combine the WRI way tags into the existing OSM counterpart. Otherwise, the OSM way will be replaced with the WRI, keeping the OSM way history.
3. Users involved in this import will validate the data to be imported, and correct the errors they may still have. Typical ones are duplicated nodes, not connected highways in crossroads, two close parallel segments that should be only one, very short few-metres segments that should be deleted, etc.
The workflow will follow these general steps:
1) One of the importing users chooses one of tasks of one of the Tasking Manager projects (there is one for each country).
2) S/he downloads the OSM data in one layer, and the WRI data for that tile in a different layer.
3) S/he then opens the aerial imagery layers we have for that area. In case there are enough GPS traces, s/he can align the imagery layers.
4) S/he then corrects the errors in the WRI data layer, like duplicated nodes, crossing ways and ways not joined to each other. Those ways that are wrong will be corrected or, in some cases, deleted and not imported. If there is any segment already present in OSM, we will conflate them as it was already explained.
5) Next, we will merge the WRI data layer with the OSM data layer and proceed to join and combine ways to finish the task.
6) Before we consider the task finished, we will check once again for any errors and warnings in the merged layer. New errors that may appear are lack of bridges or fords in river crossings.
7) Last, we upload the data with the special OSM import account.
Reverse plan
In case of any trouble, JOSM reverter will be used.
Conflation
Conflation has already been explained in the workflow section above. It will deal mainly with joining WRI roads to other OSM roads.