Import WRI Congo Basin Forestry Roads

From OpenStreetMap Wiki
Jump to: navigation, search

Goals

The World Resources Institute has released forestry road data for a set of 6 countries in Central Africa to be imported into OSM. The involved countries are: Cameroon, Equatorial Guinea, Central African Republic, Gabon, Democratic Republic of Congo and Republic of Congo.

The goal is to manually import all the road segments not yet present in OSM, joining them with the existing highways.

For this purpose, a detailed workflow will be followed by all users involved in this task.

Schedule

  1. Preparation, discussion - due to start February 9th 2015.
  2. Import - expected to start any time after the community has solved any issues, doubts or concerns about this import.

Import Data

Data description

The original data is in shapefile format. The accuracy of the data varies: some segments are quite correct and can be imported with little correction, whereas some others have to be corrected manually before upload or being used as a guide to map the actual road.

Methodologies for production process of source data was the same across all congo basin countries. Detailed on the origin and production process for the source data can be found here.

Original tags vary for each country, but are quite similar. The proposed OSM tagging tries to keep consistency across all the countries.

Original files have the following number of objects:

Number of objects
Country Number of nodes Number of ways
Cameroon 131,747 8,551
Equatorial Guinea 10,185 239
Central African Republic 16,803 1,823
Gabon 445,015 10,378
Democratic Republic of Congo 2,630 973
Republic of Congo 41,302 1,764
TOTAL 647,682 23,728

Background

ODbL Compliance verified: YES

WRI has given full authorization for the use of their data with the standard authorization document of the Humanitarian OpenStreetMap Team (HOT). A scan of the document can be found here.

Import Type

As the quality of the data varies, some ways can be imported with few corrections, but others will rather be used as a guide for tracing a more accurate one, transferring the tags from the WRI segment to the new one. Incorrect segments or segments will be ignored (deleted), whereas segments that overlap with OSM data will be conflated.

The import will be carried out by users who will read and understand the workflow wiki set for this import. For each of the 6 countries, the process will be done with the help of a HOT Tasking Manager project, where every mapper will download the OSM data and the WRI data into two different layers, so they can correct the WRI data and conflate it with the OSM data before they upload it to the OSM database.

Data Preparation

Data Reduction & Simplification

The data is originally in shapefile format, in only one file for each of the 6 countries. You can download the original files here.

From this files, we extracted the forestry relevant data. We further deleted all segments that have been already mapped in OSM.

Data has been corrected of errors, duplications, overlaps, etc., but it will be checked again during the import process.

Tagging Plans Country by Country

Cameroon
WRI Tag OSM Tag Comments
Shape_Leng=* Not tagged because not relevant
code=* Not tagged because not relevant
indicatif=* Not tagged because not relevant
last_edi_1=* Not tagged because not relevant
last_edite=* Not tagged because not relevant
n_origin=* start_date=before YYYY-MM-DD Original values are like YYYYMMDD
n_yrXXXX=* (XXXX=9900, 0001, 0102,..., 0910) All are checked together to see if abandoned, and if yes, abandoned:date=* with earliest year it is abandoned. Values are Abandonned road, Active road, Cloudy, No data and Non active road
objectid=* Not tagged because not relevant
prioritair=* Not tagged because not relevant
roadtype=Primary logging road highway=track + access=forestry If road is abandoned, then abandoned:highway=track instead of highway=track
roadtype=Secondary logging road highway=track + access=forestry If road is abandoned, then abandoned:highway=track instead of highway=track
roadtype=Forest road highway=track + access=forestry If road is abandoned, then abandoned:highway=track instead of highway=track
roadtype=Plantation road highway=track + access=agricultural If road is abandoned, then abandoned:highway=track instead of highway=track
shape_STLe=* Not tagged because not relevant
type_route=* Not tagged because not relevant
ALL source=Landsat;WRI


Equatorial Guinea
WRI Tag OSM Tag Comments
Toponimia=na Not tagged because not relevant
Categoría=Pista forestal See dscripció=* further down
Cod_WRI=* start_date=before YYYY 4 values: if 0.0 not imported, if 2001.0 YYYY=2002, if 2002.0 YYYY=2003 and if 2003.0 YYYY=2004
Comentario=na Not tagged because not relevant
dscripció=* All values, with Categoría=Pista forestal, give us access=forestry 5 values are Carretera privada principal, Carretera privada secundaria, Carretera privada terciaria, Carretera pública terciaria and Carretera secundaria
Estado2012=* Not tagged because not relevant
Fecha_act=* source:date=YYYY 2 values: 2012 and 2013
Long_km=* Not tagged because not relevant
Orig_datos=* Not tagged because not relevant
Sce_geo=* source:geometry=INDEFOR-AP cartografía
Shape_Leng=* Not tagged because not relevant
Tipo_carrt=No asfaltada surface=unpaved
Usage=* If No activa, abandoned:highway=track, else highway=track
ALL source=Landsat;WRI


Central African Republic
WRI Tag OSM Tag Comments
SHAPE_STLe=* Not tagged because not relevant
Shape_Leng=* Not tagged because not relevant
annee_ferm=* abandoned:date=before YYYY 4 values: if 0 not imported, if 2009 YYYY=2010, if 2010 YYYY=2011 and if 2012 YYYY=2013
annee_ouv=* start_date=before YYYY 6 values: if 0 not imported, if 2000 YYYY=2001, if 2008 YYYY=2009, if 2009 YYYY=2010, if 2010 YYYY=2011 and if 2011 YYYY=2012
created_da=* Not tagged because not relevant
created_us=* Not tagged because not relevant
last_edi_1=* Not tagged because not relevant
last_edite=* Not tagged because not relevant
long_km=* Not tagged because not relevant
source=* Not tagged because not relevant
statu_rou=* If statu_rou=ouvert => highway=track. If statu_rou=ferme => abandoned:highway=track If this tag is not present, then highway=track
type_route=* 5 values: if 7, 8, 9 or 10, then access=forestry. If 11, access=agricultural.
type_surf=terre surface=unpaved
ALL source=Landsat;WRI


Gabon
WRI Tag OSM Tag Comments
Annee=* start_date=before YYYY 11 values: if 0.0 or 10431.0 or 206.0 not imported, if 2004.0 YYYY=2005, if 2005.0 YYYY=2006, if 2006.0 YYYY=2007, if 2007.0 YYYY=2008, if 2008.0 YYYY=2009, if 2011.0 YYYY=2012, if 2012.0 YYYY=2013, and if 2013.0 YYYY=2014
CODE=* Not tagged because not relevant 7 values: 0, R1, R2, R3, R4, R5 and RES/Rt
CODE_BD=* highway=track + access=forestry. In case it's abandoned according to Etat_YYYY=* tags: abandoned:highway=track + access=forestry. If together with Nature=* it will be tagged according to that tag (see Nature=* further down) 1 value only: ROUTE D'EXPLOITATION
Etat_YYYY=* (YYYY=2004, 2005, 2006, 2007 and 2008) All are checked together to see if abandoned, and if yes, abandoned:date=* with earliest year it is abandoned. Values are Abandonné (or similar with typos), Fréquenté (or similar with typos) and, for Etat_2007=*, there is a third value: No data
Id=* Not tagged because not relevant Same as Annee=*, but without the .0 at the end.
Long_km=* Not tagged because not relevant
Nature=Aviation aeroway=runway or aeroway=taxiway As they total 8 segments, they will be tagged manually as either runway or taxiway
Nature=Gares railway=rail + {{ta<different>g|service|spur}}
Nature=Mines highway=track + access=private
Nature=Plantation highway=track + access=agricultural
Shape_Leng=* Not tagged because not relevant
Type_Image=* Not tagged because not relevant
ALL source=Landsat;WRI
All segments without highway=track nor abandoned:highway=track nor Nature=* highway=track


Democratic Republic of Congo
WRI Tag OSM Tag Comments
Date_orig=* Not tagged because not relevant Date of reference satellite image
Shape_Leng=* Not tagged because not relevant
categorie=Forestière highway=track + access=forestry
categorie=Chemin de Plantation highway=track + access=agricultural
categorie=* If this tag is not present: highway=track
code_wri=* Not tagged because not relevant
dat_maj=* Not tagged because not relevant same as Date_orig=*
dat_obs=* start_date=before YYYY-MM-DD Date road was first digitized
long_km=* Not tagged because not relevant
long_m=* Not tagged because not relevant
num_ccf=* Not tagged because not relevant
num_ga=* Not tagged because not relevant Code for logging concession road falls into
org_donnee=* Not tagged because not relevant
sce_geo=* source=WRI;DIAF;Africover or source=WRI;DIAF;Société Trans M or source=WRI;DIAF, depending on the value 3 values: Africover, Société Trans M and DIAF.
sce_sem=* Not tagged because not relevant Organization which compiled/ complete data: DIAF.
type=* Not tagged because not relevant 4 values: Exploitation Sec, Futur Princ, Futur Sec, Public


Republic of Congo
WRI Tag OSM Tag Comments
Annee=* start_date=before YYYY 8 values: 0, 1988, 2000, 2001, 2002, 2003, 2006 and 2007. If 0 not imported. For the rest of the values: XXXX -> YYYY=XXXX+1
CDE_VCOM=YYYY.0 Not tagged because not relevant
DESCRIPTIO=Route forestière principale access=forestry
DESCRIPTIO=Route forestière secondaire access=forestry
DESCRIPTIO=Route d'exploitation access=forestry
DESCRIPTIO=Route agro industrie access=agricultural
ETAT=* If ETAT=Abandonnée, then abandoned:highway=track. Else, highway=track 2 values: Abandonnée and En service
Long_km=* Not tagged because not relevant
Longueur=* Not tagged because not relevant
Periode=* When together with source:geometry=Aster and not year 1899, then source:date=YYYY-MM-DD Periode is the source date of Aster aerial imagery
SOURCE=* source:geometry=* . 3 values: source:geometry=Aster, source:geometry=DMCii and source:geometry=LandSat
Shape_Leng=* Not tagged because not relevant
Usage=* Not tagged because not relevant 3 values: Nulle or nulle, Permanente and Temporaire.
ALL source=WRI

Changeset Tags

We will use the following changeset tags:

where imageryProvider=Bing, Mapbox or whatever imagery used while importing the data, NUMBEROFPROJECT being the Tasking Manager project number and NAME_OF_COUTRY be substituted by the country code: CMR for Cameroon, GNQ for Equatorial Guinea, etc.

Data Transformation

Each of the 6 files in shapefile format were opened with JOSM + Open Data plugin, and saved in osm format. After this, each one was transformed into a new osm file with the proposed tags, using an ad-hoc awk script for each one.

You can download those final files and scripts here.

Data Merge Workflow

Team Approach

Import will be undertaken by mappers trained for this task, using an import specific OSM user account and following a [WORKFLOW LINK detailed workflow with screenshots], that shows all the steps to follow, with the goal of best keeping the consistency and correctness during all the process.

References

Following the imports guidelines, this import will be discussed first in the local mailing list for R. D. of Congo, Cameroon and Central African Rep., and in the HOT mailing list for Gabon, Equatorial Guinea and Rep. of Congo, as they don't have a local OSM chapter.

With the inputs from those lists, we will open a thread in the imports list.

Workflow

The full workflow wiki can be checked here.

This workflow will deal with the following issues:

1. In areathat will be s with high resolution aerial imagery, users will check the accuracy of each segment against that imagery, and will correct it or trace a new one using the WRI segment as a guide if accuracy is poor.

2. Conflating with the existing OSM highways: As explained before, the roads in the WRI dataset that conflict with others already in OSM have been removed already. If one user finds a segment conflicting with an OSM highway, he/she will check which one is more accurate. In case the OSM way is better, s/he will combine the WRI way tags into the existing OSM counterpart. Otherwise, the OSM way will be replaced with the WRI, keeping the OSM way history.

3. Users involved in this import will validate the data to be imported, and correct the errors they may still have. Typical ones are duplicated nodes, not connected highways in crossroads, two close parallel segments that should be only one, very short few-metres segments that should be deleted, etc.

The workflow will follow these general steps:

1) One of the importing users chooses one of tasks of one of the Tasking Manager projects (there is one for each country).

2) S/he downloads the OSM data in one layer, and the WRI data for that tile in a different layer.

3) S/he then opens the aerial imagery layers we have for that area. In case there are enough GPS traces, s/he can align the imagery layers.

4) S/he then corrects the errors in the WRI data layer, like duplicated nodes, crossing ways and ways not joined to each other. Those ways that are wrong will be corrected or, in some cases, deleted and not imported. If there is any segment already present in OSM, we will conflate them as it was already explained.

5) Next, we will merge the WRI data layer with the OSM data layer and proceed to join and combine ways to finish the task.

6) Before we consider the task finished, we will check once again for any errors and warnings in the merged layer. New errors that may appear are lack of bridges or fords in river crossings.

7) Last, we upload the data with the special OSM import account.

Reverse plan

In case of any trouble, JOSM reverter will be used.

Conflation

Conflation has already been explained in the workflow section above. It will deal mainly with joining WRI roads to other OSM roads.