Facebook AI-Assisted Road Tracing

From OpenStreetMap Wiki
(Redirected from AI-Assisted Road Tracing)
Jump to navigation Jump to search

Facebook have been running a project to perform AI-Assisted Road Tracing within OpenStreetMap. This has taken place in several phases, and involved local OpenStreetMap communities in Thailand and Indonesia.

In 2016, Facebook started researching how various machine learning algorithms could be integrated into a mapping workflow. At State of the Map 2016 Facebook shared initial results on how one could use AI and new mapping techniques, such as advanced deep neural net models, to detect roads from satellite imagery. We have since received many requests to share our process and masks. Our goal is to collaborate with the OpenStreetMap community to create quality road data, validated by trained mappers and local OSM communities. Please send questions to osm@fb.com.

For more information on the project please also check out Mapwith.ai and the github page Blacktocat.svg Facebook OSM Data Team

Schedule

Partnering with the local OSM communities, the pilot mapping countries were Thailand followed by Indonesia. They chose these countries because of their relative low road coverage on OSM.

Ongoing Projects

Vietnam

  • Work began January 21st, 2020
  • Changeset: #mapwithai #Vietnam

Tanzania

  • Road mapping will begin once Malaysia is completed
    Changeset: #mapwithai #Tanzania

India

Completed Projects

Thailand

  • June 1, 2017 to December 21, 2018
  • Stats for total project length (including community contributions during the time frame):
    • 956,381 New ways added
    • 515,306 km of roads added

Indonesia

  • Road Network Changes from March 1, 2018 to September 24, 2019
    • 1,095,034 new ways added
    • 474,409 km of roads added
  • Stats for total project length (including community contributions during the time frame)
    • 1,711,470 new ways added
    • 614,328 km of roads added

Haiti

  • December 23, 2019 - December 31, 2019
  • We added missing roads to area around the city of Leogane at the request of MotoMeds Haiti.

Malaysia

  • Ongoing since September 11, 2019
    Changeset: #mapwithai #Malaysia

Data Processing

DigitalGlobe source imagery

DigitalGlobe's Vivid+ product provides high-resolution (50cm / pixel, or zoom 18) color-corrected cloud-free satellite imagery with full country coverage. We now have license to extract road data from the imagery and contribute to OSM.

The following images show the benefits of using current high-resolution imagery. For example the quality of satellite image currently available to the OSM community for Mu Ban Don Sak Thani, Thailand is not optimal for mapping.

Mu Ban Don Sak Thani of Thailand. Vivid+ on left, Bing Aerial on right, at zoom 14, which is Bing Aerial's max zoom for the city.

Figure 1.png

The full resolution goes to zoom 18.

Figure 2.png

Another example of Ko Samui where Bing has zoom 18 but it's heavily covered by clouds.

Figure 3.png

Road Extraction

We will only contribute roads to OSM. We use the latest deep learning algorithms to assist road extraction and mapping. Here is an example of a prediction using deep learning for an area in Thailand.

Data Processing Image Example.png

Data Processing Image Example 2.png

We then apply a series of post processing steps to generate .osm file that can be loaded into JOSM or our modified iD for manual correction and enhancement. The images below show the .osm file in our iD, where the grey road is an existing road on OSM, and green roads are new.

Data Processing Image Example 3.png

Data Processing Image Example 4.png

Submission Process

The submission process consists of the following steps summarized in the image below.

Project Workflow.png

(1) We use the OSM Tasking manager to divide up tasks and create a flow of editing and validation for the Facebook mapping team.

(2) The Mapper picks a task and loads an .osm/.xml file generated from the post processing step into our enhanced version of iD (this includes both the roads generated by us and the current OSM data).

(3) The Mapper evaluates the roads we generated (styled in a different color) and inspect them for issues like crossing highways, disconnected roads, incorrect intersections, short road stubs, road types, etc., and fixes these issues manually one by one, using DigitalGlobe satellite imagery.

(4) Our modified iD tool is equipped with data validation functionalities similar to JOSM and osmlint. This allows us to check for quality and conflicts with current OSM data prior to submission. Some examples of what we check for include:

  • Isolated roads that are not connected to rest of the map.
  • Roads crossing waterways.
  • Roads with repeating node IDs.
  • Roads with possible connections after a short extension.
  • Self-intersecting ways.

You can watch the video example of the various data integrity editing features here.

(5) In case of conflicts between our newly added roads and other OSM contributors' changes, our mappers will almost always choose “keep their edits.” If we end up changing another mapper's edits, (for example, to align better with the latest version of DG satellite imagery) we will leave detailed notes to explain why. 

(6) Until all errors are fixed, Mappers will not be able t upload to OSM. Here is an example of a highlighted issue prompting the mapper to fix it.

(7) After fixing all the issues detected in the iD tool, the Mapper clicks the “Save local” button to save their edited roads locally for validation.

(8) A Validator then goes into the same task to verify that the data looks correct by re-validating the data, makes necessary changes as needed, and clicks “Submit” to finally upload the task to OSM. 

(9) The Validator then conducts a post-submission review of the submitted edits in JOSM. If any errors are detected, the validator addresses the errors and submits necessary changes. A custom JOSM paint style is used to help distinguish our submissions from community submissions, so that all roads with the import=yes tag are highlighted in green. Some examples for what we check for include: 

  • Short Unclassified ways (between 10-60 meters)
  • Highway crossing waterway without a bridge tag
  • Highway crossing buildings
  • Overlapping ways
  • Unspecific highway tags

(10) We will continue to watch all comments made on our edits, reply to them in a timely fashion. We are happy to incorporate feedback and local expertise into our validation process and future mapping behaviors. 

Data Preparation

Road Type Tagging

During the submission process, our mappers will assign and fix road types according to the general highway tag guidelines of OSM http://wiki.openstreetmap.org/wiki/Key:highway

Before moving to a new location, we will work with mappers from local OSM community in each country to verify our highway tagging method and make sure they align with local conventions.

Some other methods we use to tag roads include:

  • Referring to local Wiki guides for each country.
  • Looking at commonly used tags in the local area of mapping.
  • When connecting to current OSM roads on the map we try to use the same tag where it makes sense to do so.
  • Referring to local community for advice.

Changeset Tags

We use the following tags:

Changeset Tags:

We use the above changeset tags in every import changeset, where the '#country' tag is the country of the submission. For Thailand the tags are '#nsroadimport #thailand'.

Element Tags:

  • source = digitalglobe
  • import = yes

The 'source' tag indicates that road geometries are based on DigitalGlobe satellite imagery.

Data Merge Workflow

Team Approach

This import (data integration) will be done through a Tasking Manager, by engineers and a group of selected trained map editors. Among the skills required:

  • Good experience with iD and JOSM.
  • Familiarity with editing norms: how to merge nodes (M), join ways (J), combine ways (C) and un-glueing (G)
  • S/he knows how to handle conflicts with both iD tool and JOSM.

The map editing team consists of Mappers and Validators, and their accounts are listed as below.

Mappers:

Name Username Role Link
Zvone RVR001 Validator https://www.openstreetmap.org/user/RVR001
Jeff RVR002 QA https://www.openstreetmap.org/user/RVR002
Yunzhi RVR004 QA https://www.openstreetmap.org/user/RVR004
Charmaine RVR012 QA https://www.openstreetmap.org/user/RVR012
Dylan VLD032 Mapper https://www.openstreetmap.org/user/VLD032
Katherine VLD035 Validator https://www.openstreetmap.org/user/VLD035
Jesspher VLD062 Mapper https://www.openstreetmap.org/user/VLD062
Ming VLD066 Mapper https://www.openstreetmap.org/user/VLD066
Jacob VLD067 Mapper https://www.openstreetmap.org/user/VLD067
Christopher VLD073 Mapper https://www.openstreetmap.org/user/VLD073
Michael VLD075 Mapper https://www.openstreetmap.org/user/VLD075
Lewei VLD076 Mapper https://www.openstreetmap.org/user/VLD076
Andre VLD077 Mapper https://www.openstreetmap.org/user/VLD077
Courtney VLD079 Mapper https://www.openstreetmap.org/user/VLD079
Henry VLD080 Mapper https://www.openstreetmap.org/user/VLD080
Adrian VLD081 Validator https://www.openstreetmap.org/user/VLD081
Gwendolyn VLD082 Mapper https://www.openstreetmap.org/user/VLD082
Julia VLD083 Mapper https://www.openstreetmap.org/user/VLD083
Joshua VLD085 Mapper https://www.openstreetmap.org/user/VLD085
Hector VLD087 Mapper https://www.openstreetmap.org/user/VLD087
Steve VLD089 Mapper https://www.openstreetmap.org/user/VLD089
Tukui VLD090 Mapper https://www.openstreetmap.org/user/VLD090
Yanitza VLD092 Mapper https://www.openstreetmap.org/user/VLD092
Je-Ni VLD093 Mapper https://www.openstreetmap.org/user/VLD093
Amin VLD094 Mapper https://www.openstreetmap.org/user/VLD094
Kayla VLD096 Mapper https://www.openstreetmap.org/user/VLD096
Matthew VLD097 Mapper https://www.openstreetmap.org/user/VLD097
Narendar VLD109 Mapper https://www.openstreetmap.org/user/VLD109
Teesta VLD124 Mapper https://www.openstreetmap.org/user/VLD124
Gunapati VLD126 Mapper https://www.openstreetmap.org/user/VLD126
Nayani VLD127 Validator https://www.openstreetmap.org/user/VLD127
Madduluri VLD129 Mapper https://www.openstreetmap.org/user/VLD129
Srinivas VLD130 Mapper https://www.openstreetmap.org/user/VLD130
Chittipolu VLD131 Mapper https://www.openstreetmap.org/user/VLD131
Mahesh VLD132 Mapper https://www.openstreetmap.org/user/VLD132
Jammula VLD133 Mapper https://www.openstreetmap.org/user/VLD133
Shubham VLD134 Mapper https://www.openstreetmap.org/user/VLD134
Meka VLD135 Mapper https://www.openstreetmap.org/user/VLD135
Venkata VLD136 Mapper https://www.openstreetmap.org/user/VLD136
Adabala VLD138 Mapper https://www.openstreetmap.org/user/VLD138
Ashok VLD139 Mapper https://www.openstreetmap.org/user/VLD139
Saicharan VLD140 Mapper https://www.openstreetmap.org/user/VLD140
Soundarya VLD141 Mapper https://www.openstreetmap.org/user/VLD141
Rampeesa VLD142 Mapper https://www.openstreetmap.org/user/VLD142
Gumpili VLD143 Mapper https://www.openstreetmap.org/user/VLD143
Abdul VLD144 Mapper https://www.openstreetmap.org/user/VLD144
Vishal VLD145 Mapper https://www.openstreetmap.org/user/VLD145
Bokam VLD146 Mapper https://www.openstreetmap.org/user/VLD146
Joanna VLD150 Mapper https://www.openstreetmap.org/user/VLD150
Jami VLD151 Mapper https://www.openstreetmap.org/user/VLD151
Brenna VLD152 Mapper https://www.openstreetmap.org/user/VLD152
Johnny VLD153 Mapper https://www.openstreetmap.org/user/VLD153
Chris VLD154 Mapper https://www.openstreetmap.org/user/VLD154
Ethan VLD155 Mapper https://www.openstreetmap.org/user/VLD155
Sherry VLD156 Mapper https://www.openstreetmap.org/user/VLD156
Callista VLD157 Mapper https://www.openstreetmap.org/user/VLD157
Mark VLD158 Mapper https://www.openstreetmap.org/user/VLD158
Anthony VLD159 Mapper https://www.openstreetmap.org/user/VLD159
Kohl VLD160 Mapper https://www.openstreetmap.org/user/VLD160
Elisa VLD161 Mapper https://www.openstreetmap.org/user/VLD161
Christopher VLD162 Mapper https://www.openstreetmap.org/user/VLD162
Vladimir VLD163 Mapper https://www.openstreetmap.org/user/VLD163
Meng VLD164 Mapper https://www.openstreetmap.org/user/VLD164
Yemi VLD165 Mapper https://www.openstreetmap.org/user/VLD165
Jordan VLD166 Mapper https://www.openstreetmap.org/user/VLD166
Elliot VLD167 Mapper https://www.openstreetmap.org/user/VLD167
Shradha VLD200 Team Lead https://www.openstreetmap.org/user/VLD200
Mourya VLD201 Validator https://www.openstreetmap.org/user/VLD201
Pulumamidi VLD202 Validator https://www.openstreetmap.org/user/VLD202
Madduluri VLD203 Validator https://www.openstreetmap.org/user/VLD203
Rasamalla VLD204 Validator https://www.openstreetmap.org/user/VLD204
Neela VLD205 Validator https://www.openstreetmap.org/user/VLD205
Saisharath VLD206 Validator https://www.openstreetmap.org/user/VLD206
Arusula VLD207 Mapper https://www.openstreetmap.org/user/VLD207
Oraganti VLD208 Mapper https://www.openstreetmap.org/user/VLD208
Madagoni VLD209 Mapper https://www.openstreetmap.org/user/VLD209
P.Ravi VLD210 Mapper https://www.openstreetmap.org/user/VLD210
Sravan VLD211 Mapper https://www.openstreetmap.org/user/VLD211
Malka VLD212 Mapper https://www.openstreetmap.org/user/VLD212
Vasireddy VLD213 Mapper https://www.openstreetmap.org/user/VLD213
Marupally VLD214 Mapper https://www.openstreetmap.org/user/VLD214
Darji VLD215 Mapper https://www.openstreetmap.org/user/VLD215
Konda VLD216 Mapper https://www.openstreetmap.org/user/VLD216
Mir VLD217 Mapper https://www.openstreetmap.org/user/VLD217
Priya VLD218 Mapper https://www.openstreetmap.org/user/VLD218
Ramanyam VLD219 Mapper https://www.openstreetmap.org/user/VLD219
Jaddu VLD220 Mapper https://www.openstreetmap.org/user/VLD220
Syam VLD221 Mapper https://www.openstreetmap.org/user/VLD221
Shrirame VLD222 Mapper https://www.openstreetmap.org/user/VLD222
Harika VLD223 Mapper https://www.openstreetmap.org/user/VLD223
Ramya VLD224 Mapper https://www.openstreetmap.org/user/VLD224
B. Navya VLD225 Mapper https://www.openstreetmap.org/user/VLD225
Pavirala VLD226 Mapper https://www.openstreetmap.org/user/VLD226
Sachin VLD227 Mapper https://www.openstreetmap.org/user/VLD227
Aneesha VLD228 Mapper https://www.openstreetmap.org/user/VLD228
Ravali VLD229 Mapper https://www.openstreetmap.org/user/VLD229
Shivanl VLD230 Mapper https://www.openstreetmap.org/user/VLD230
Roshini VLD231 Mapper https://www.openstreetmap.org/user/VLD231


We are not bots :) The usernames (VLD001, RVR001, etc.) were made to help our internal process where we have paired Mappers and Validators to ensure quality. Please feel free to send us questions individually about edits or email all of us at osm@fb.com.

Sub-contracted Teams

Humanitarian OpenStreetMap Team (HOT)

For mapping Indonesia, Facebook has subcontracted to HOT employees, as they have an office in Jakarta; in order to better coordinate with the local community and improve the road network of Indonesia. The team is comprised of:

Name Username Role Link
Adiatma Adiatma IRM-RV Team Lead https://www.openstreetmap.org/user/Adiatma%20IRM-RV
Akrimullah Akrimullah IRM-RV QA https://www.openstreetmap.org/user/Akrimullah%20IRM-RV
Ficky Ficky IRM-ED QA [osm:user/Fanny IRM-ED https://www.openstreetmap.org/user/Ficky%20IRM-ED]
Hanif Hanif IRM-RV QA https://www.openstreetmap.org/user/Hanif%20IRM-RV
Riyadi Riyadi IRM-ED QA https://www.openstreetmap.org/user/Riyadi%20IRM-ED
Aglis Aglis IRM-ED Mapper https://www.openstreetmap.org/user/Aglis%20IRM-ED
Anisa Anisa IRM-ED Mapper https://www.openstreetmap.org/user/Anisa%20IRM-ED
Arma Arma IRM-ED Mapper https://www.openstreetmap.org/user/Arma%20IRM-ED
Diah Diah IRM-ED Mapper https://www.openstreetmap.org/user/Diah%20IRM-ED
Diana Diana IRM-ED Mapper https://www.openstreetmap.org/user/Diana%20IRM-ED
Elisabeth Elisabeth IRM-ED Mapper https://www.openstreetmap.org/user/Elisabeth%20IRM-ED
Elyana Elyana IRM-ED Mapper https://www.openstreetmap.org/user/Elyana%20IRM-ED
Fadlilaa Fadlilaa IRM-ED Mapper https://www.openstreetmap.org/user/Fadlilaa%20IRM-ED
Fanny Fanny IRM-ED Mapper https://www.openstreetmap.org/user/Fanny%20IRM-ED
Fauzan Fauzan IRM-ED Mapper https://www.openstreetmap.org/user/Fauzan%20IRM-ED
Qomar Qomar IRM-ED Mapper https://www.openstreetmap.org/user/Qomar%20IRM-ED
Rizky Rizky IRM-ED Mapper https://www.openstreetmap.org/user/Rizky%20IRM-ED
Silvia Silvia IRM-ED Mapper https://www.openstreetmap.org/user/Silvia%20IRM-ED
Talitha Talitha IRM-ED Mapper https://www.openstreetmap.org/user/Talitha%20IRM-ED
Tantri Tantri IRM-ED Mapper https://www.openstreetmap.org/user/Tantri%20IRM-ED
Zainab Zainab IRM-ED Mapper https://www.openstreetmap.org/user/Zainab%20IRM-ED
Dorica doricamugusi_ImportAccount Team Lead https://www.openstreetmap.org/user/doricamugusi_ImportAccount
Aron Aron Martin_ImportAccount QA https://www.openstreetmap.org/user/Aron%20Martin_ImportAccount
Christina Christina Reuben_ImportAccount QA https://www.openstreetmap.org/user/Christina%20Reuben_ImportAccount
Dorice Dorice lucas_ImportAccount QA https://www.openstreetmap.org/user/Dorice%20lucas_ImportAccount
Neema NeemaAlphonce_ImportAccount QA https://www.openstreetmap.org/user/NeemaAlphonce_ImportAccount
Abou Abou kachongo jr_ImportAccount Mapper https://www.openstreetmap.org/user/Abou%20kachongo%20jr_ImportAccount
Asibwene asibwene_ImportAccount Mapper https://www.openstreetmap.org/user/asibwene_ImportAccount
Catherine CatherineG_ImportAccount Mapper https://www.openstreetmap.org/user/CatherineG_ImportAccount
Charles charles Frank_ImportAccount Mapper https://www.openstreetmap.org/user/charles%20Frank_ImportAccount
Damon Damon Monyo-ImportAccount Mapper https://www.openstreetmap.org/user/DamonMonyo_ImportAccount
Furaha Furaha L Mrema_ImportAccount Mapper https://www.openstreetmap.org/user/Furaha%20L%20Mrema_ImportAccount
Godfrey F Godfrey Faustine_ImportAccount Mapper https://www.openstreetmap.org/user/Godfrey%20Faustine_ImportAccount
Godfrey H Godfrey Hussein_ImportAccount Mapper https://www.openstreetmap.org/user/Godfrey%20Hussein_ImportAccount
Jamal JAMAL AMANI_ImportAccount Mapper https://www.openstreetmap.org/user/JAMAL%20AMANI_ImportAccount
Nuru NURU ATHUMANI_ImportAccount Mapper https://www.openstreetmap.org/user/NURU%20ATHUMANI_ImportAccount
Samwel Samwel Kyando_ImportAccount Mapper https://www.openstreetmap.org/user/Samwel%20Kyando_Import%20Account
Alpha Alpha Ndahoze_ImportAccount Mapper https://www.openstreetmap.org/user/Alpha%20Ndahoze_ImportAccount
Neema CatherineG_ImportAccount Mapper https://www.openstreetmap.org/user/CatherineG_ImportAccount
William B WillliamBonya_ImportAccount Mapper https://www.openstreetmap.org/user/william%20bonya_ImportAccount
William WilliamMponeja_ImportAccount Mapper https://www.openstreetmap.org/user/William%20Mponeja_ImportAccount
Zaituni ZaituniAwadhi_ImportAccount Mapper https://www.openstreetmap.org/user/Zaituni%20Awadhi_ImportAccount
Zaina Zaina Rashidi_ImportAccount Mapper https://www.openstreetmap.org/user/Zaina%20Rashidi%20ImportAccount


Inactive users

Name Username Link
Adam Adam IRM-ED https://www.openstreetmap.org/user/Adam%20IRM-ED
Aude Aude IRM-ED https://www.openstreetmap.org/user/Aude%20IRM-ED
atmahadi atmahadi IRM-ED https://www.openstreetmap.org/user/atmahadi%20IRM-ED
Dary Dary IRM-ED https://www.openstreetmap.org/user/Dary%20IRM-ED
Dennis Dennis IRM-ED https://www.openstreetmap.org/user/Dennis%20IRM-ED
Deo Deo IRM-ED https://www.openstreetmap.org/user/Deo%20IRM-ED
Diah Suci Diah Suci IRM-ED https://www.openstreetmap.org/user/Diah%20Suci%20IRM-ED
Fatri Fatri IRM-ED https://www.openstreetmap.org/user/Fatri%20IRM-ED
Fathin Fathin IRM-ED https://www.openstreetmap.org/user/Fathin%20IRM-ED
Iko Iko IRM-ED https://www.openstreetmap.org/user/Iko%20IRM-ED
Irwan Irwan IRM-ED https://www.openstreetmap.org/user/Irwan%20IRM-ED
Muty Muty IRM-ED https://www.openstreetmap.org/user/Muty%20IRM-ED
Ratry Ratry IRM-ED https://www.openstreetmap.org/user/Ratry%20IRM-ED
Wulan Wulan IRM-RV https://www.openstreetmap.org/user/Wulan%20IRM-RV
Yeni Yeni IRM-RV https://www.openstreetmap.org/user/Yeni%20IRM-RV
Edger Edger_ImportAccount https://www.openstreetmap.org/user/Edger_ImportAccount
Erasto ERASTO ELIUD_ImportAccount https://www.openstreetmap.org/user/ERASTO&20ELIUD_ImportAccount
Gwantwa Gwantwa_Andrew_ImportAccount https://www.openstreetmap.org/user/Gwantwa_Andrew_ImportAccount
Janet janethmlinga_ImportAccount https://www.openstreetmap.org/user/janethmlinga_ImportAccount
Mariamu Mariamu Crispo_ImportAccount https://www.openstreetmap.org/user/Mariamu%20Crispo_ImportAccount
David RVR003 https://www.openstreetmap.org/user/RVR003
Adrian RVR005 https://www.openstreetmap.org/user/RVR005
Stefani RVR006 https://www.openstreetmap.org/user/RVR006
Mohamed RVR007 https://www.openstreetmap.org/user/RVR007
Sam RVR009 https://www.openstreetmap.org/user/RVR009
Jaclyn VLD001 https://www.openstreetmap.org/user/VLD001
Annie VLD002 https://www.openstreetmap.org/user/VLD002
Alexandra VLD003 https://www.openstreetmap.org/user/VLD003
Sheffield VLD004 https://www.openstreetmap.org/user/VLD004
Mei VLD005 https://www.openstreetmap.org/user/VLD005
Joseph VLD006 https://www.openstreetmap.org/user/VLD006
Kurt VLD007 https://www.openstreetmap.org/user/VLD007
Pablo VLD008 https://www.openstreetmap.org/user/VLD008
Yoko VLD009 https://www.openstreetmap.org/user/VLD009
Klavdija VLD010 https://www.openstreetmap.org/user/VLD010
Michael VLD011 https://www.openstreetmap.org/user/VLD011
Paris VLD012 https://www.openstreetmap.org/user/VLD012
Kailei VLD013 https://www.openstreetmap.org/user/VLD013
Jen VLD014 https://www.openstreetmap.org/user/VLD014
Joseph VLD015 https://www.openstreetmap.org/user/VLD015
Gemma VLD016 https://www.openstreetmap.org/user/VLD016
Lena VLD018 https://www.openstreetmap.org/user/VLD018
Elizabeth VLD019 https://www.openstreetmap.org/user/VLD019
Alayna VLD025 https://www.openstreetmap.org/user/VLD025
Louis VLD027 https://www.openstreetmap.org/user/VLD027
Ismael VLD028 https://www.openstreetmap.org/user/VLD028
Savanna VLD033 https://www.openstreetmap.org/user/VLD033
Christina VLD034 https://www.openstreetmap.org/user/VLD034
Jacob VLD039 https://www.openstreetmap.org/user/VLD039
Jonathan VLD042 https://www.openstreetmap.org/user/VLD042
Casey VLD044 https://www.openstreetmap.org/user/VLD044
Elena VLD050 https://www.openstreetmap.org/user/VLD050
Linzi VLD053 https://www.openstreetmap.org/user/VLD053
Ryan VLD054 https://www.openstreetmap.org/user/VLD054
Yonis VLD055 https://www.openstreetmap.org/user/VLD055
Luxinaree VLD056 https://www.openstreetmap.org/user/VLD056
Katalin VLD057 https://www.openstreetmap.org/user/VLD057
John VLD058 https://www.openstreetmap.org/user/VLD058
Nick VLD059 https://www.openstreetmap.org/user/VLD059
Guowei VLD063 https://www.openstreetmap.org/user/VLD063
Derrick VLD070 https://www.openstreetmap.org/user/VLD070
Johanna VLD078 https://www.openstreetmap.org/user/VLD078
Ashley VLD088 https://www.openstreetmap.org/user/VLD088
Rena VLD017 https://www.openstreetmap.org/user/VLD017
Tina VLD020 https://www.openstreetmap.org/user/VLD020
Ian VLD023 https://www.openstreetmap.org/user/VLD023
Maureen VLD026 https://www.openstreetmap.org/user/VLD026
Sophie VLD030 https://www.openstreetmap.org/user/VLD030
Olympia VLD036 https://www.openstreetmap.org/user/VLD036
Rowen VLD040 https://www.openstreetmap.org/user/VLD040
Mike VLD041 https://www.openstreetmap.org/user/VLD041
Haley VLD043 https://www.openstreetmap.org/user/VLD043
Brady VLD045 https://www.openstreetmap.org/user/VLD045
Kaiti VLD046 https://www.openstreetmap.org/user/VLD046
Maegan VLD048 https://www.openstreetmap.org/user/VLD048
Gerum VLD052 https://www.openstreetmap.org/user/VLD052
Ryan VLD074 https://www.openstreetmap.org/user/VLD074
Jeff VLD091 https://www.openstreetmap.org/user/VLD091
Larissa VLD095 https://www.openstreetmap.org/user/VLD095
Bala VLD100 https://www.openstreetmap.org/user/VLD100
Samba VLD117 https://www.openstreetmap.org/user/VLD117
Sandeep VLD118 https://www.openstreetmap.org/user/VLD118
Shakil VLD119 https://www.openstreetmap.org/user/VLD119
Shyam VLD128 https://www.openstreetmap.org/user/VLD128
Joseph VLD072 https://www.openstreetmap.org/user/VLD072
Cameron VLD086 https://www.openstreetmap.org/user/VLD086


References

Following the imports guidelines, these imports will be discussed first in the country specific mailing list, and to get more feedback it will also be shared with the import list.

Workflow

In summary, the overall mapping workflow is:

(1) Extract grey-scale road mask images from high resolution DigitalGlobe satellite imagery.

(2) enerate road vectors based on the road mask images and merge them with latest OSM data to produce .osm files.

(3) The human mapping team does manual validation on the .osm files through our enhanced iD tool and on JOSM prior to submission (as described in the “Submission Process” above).

(4) After conflict resolution, submission, and post-submission review, work with local OSM community to complete local validation as described in the “Validation Steps” below.

Validation Steps

There will be a 4 steps validation process done by experienced OSM editors including local country Mappers.

  • Our method for each country will include starting with small number of edits to gather feedback from the local OSM community before proceeding with larger edits.
  • We then start with human validation from our Mappers as explained in the “Submission Process.”
  • Before submitting our changes, our team will verify that our updates are based on the current dataset.
  • After Submission (as described in the “Submission Process” above, a team of selected Validators and QA's conduct a second post-submission review by passing over chunks of multiple submitted tasks , region by region, by repeating the JOSM validation checks in described in Step 9 under “Submission.”
  • Our validation process primarily only includes elements that our team has contributed, which can be identified by the “import=yes” tag. Elements that our team “inherits,” meaning elements that are authored by one of our usernames but lacking the “import=yes” tag, are not typically included in post-submission validation. For more about inheritance, see footnotes.
    • We understand inherited versus contributed elements by exploring the ownership of nodes and their parent ways. We can identify inheritance based on authorship and tagging. Roads that are authored by one of our usernames, but does not include the Import=yes tag, is likely an inherited way. Because we are a team of remote mappers, using one data source, we want to avoid overstepping our contributions.
  • Community feedback on imported roads is evaluated and incorporated for final edits in this stage as well. All edits can be identified with the changeset tags above and the team will respond to any e-mails or concerns about edits within 48 hours.

To facilitate the validation process, we share our road mask as a tile service that can be used in iD tool and JOSM as a background layer. The URL template of the tile service is distributed on a per-request basis. Please contact osm@fb.com for requesting a validated URL template.

So far we have prepared road masks for selected areas in Egypt and Thailand. We will announce the availability for other countries here as they become ready.

Local OSM Community Engagement - Using the list servers and wikis we contact local OSM community members before starting to edit in a specific country. Through this we invite members to collaborate and provide valuable local insight and feedback.

Legal terms of use for our tile service: see Media:FacebookRoadMaskLicense.pdf. Information regarding DigitalGlobe Imagery: DigitalGlobe confirms the above stated license grants pertaining to DigitalGlobe satellite imagery are accurate.

To use our tile service in iD:

(1) Click the “Edit custom background” button in iD

Legal terms of use for our tile service 1.png

(2) Paste the URL template above in the input box of the pop-up dialog and click “OK”

Legal terms of use for our tile service 2.png

To use our tile service in JOSM:

(1) Click Imagery > Imagery Preferences dialog

(2) In the Imagery providers tab, in the lower right hand side of the dialog box, click on “+TMS” button

Legal terms of use for our tile service 3.png

(3) Enter the URL template in the input box and provide a name for the new layer

Legal terms of use for our tile service 4.png

(4) Once added, this new layer will show up under the Imagery list. You can select this layer to be displayed in JOSM

Legal terms of use for our tile service 5.png


FAQ's

What does AI mean?
Artificial Intelligence (AI) is training a computer to do things that require intelligence when done by humans. In this case it involves creating training data for an area by tracing out satellite imagery the same way you would when editing and feeding it to a computer. The output of this is road masks. Here is an example prediction result using deep learning for an area in Thailand.
Are there limitations with AI?
Yes, AI and algorithms still have a long way to go to figure things such as assigning tags according to OSM region specific guidelines, figure out the correct OSM relation, add bridges over waterbodies or figure out connections with crossing railways among many other things. Currently all of this is done by our mappers manually.
Are you adding roads directly to OSM without humans validating?
Every single road from the AI output will be added only after multiple rounds of human validation. After some processing of the road masks shown above we end up with a .osm file. We then treat this like remote tracing. Using iD editor and/or JOSM mappers make sure that merging with the current data has happened smoothly. They tag each road appropriately, connect them to current OSM edits, fix alignment, when necessary, add sensible changeset comments/notes and resolve conflicts before finally saving where it is merged with the current OSM data.
Why are you using iD Editor versus JOSM?
We know JOSM is efficient at what we are trying to do. In fact, we started primarily with JOSM. This said, to make our internal process more efficient, our engineers improved the iD editor so our version has comparable functionality as JOSM. We are working on making this public so we can share it with the community.
What Imagery are you using?
We are using DigitalGlobe's Vivid+ which is high-resolution (50cm / pixel, or zoom 18) color-corrected and cloud-free
Can you share DG imagery?
Our imagery is very similar to the Maxar imageries available in iD and JOSM. Our exact version used for prediction is available in RapiD as Facebook's Map With AI - Maxar Imagery
Facebook License with Digital Globe
This license is necessary to comply with our obligations to Digital Globe, and we think the benefits of supplying this data outweigh the licensing restraints. The data will be under ODBL once it has been contributed to OSM in compliance with the terms.
There is no contradiction between the OSM terms and the Facebook license with Digital Globe. Our license terms do not apply to data once contributed to OSM and termination of the agreement with Digital Globe would not impact the data that has already been contributed to OSM.
Is the Imagery free of Cloud Cover?
The intended output of +Vivid is cloud free, and in most cases, DG meets this. There are some highly cloudy parts of the world where it’s unavoidable. On average though, this product maintains less than 1% cloud cover.
Who is behind osm@fb.com?
There are 4 people behind this email. This helps us collaborate as a team and is the most efficient way for everyone to be on the same page. More people also ensures that no emails are missed. I (Drishtie Patel) as the Program Manager primarily watch this email and respond to queries along with Ming Gao and Saikat Basu.
Can the Chageset be more specific?
Yes we can absolutely change that to import=facebook-ai-____. We don't specifically have versions but can include the area or task number for example. Open to suggestions.
As for adding more details to our edits: there were no changes to current OSM data in this sample and we plan to add notes when we upload as would normally happen when using the Tasking Manager and iD. The sample shared has not been uploaded yet.
Can you Publish Source Code for ML?
We cannot share the source code for Machine Learning at this time, but we can and will share our internal tools.
Can you Publish Source Code for internal tools?
Yes. We are currently using a version of iD Editor and the HOT Tasking Manager. We are absolutely going to share our tooling. We are currently working on creating a Github where we will post soon.
Can Facebook Share other data?
This is clear violation of privacy so we cannot share this kind of information. We are exploring ways to crowdsource road names for OSM and will share that as we have a more concrete plan.
Why did you choose Thailand?
Facebook has a high number of users in this country and we would like to improve that map for this community. It is also our mission as a company is to make the world more open and connected and one way we can do is by filling in the missing gaps on the map. We also saw a strong OSM community that we could learn from and engage with while we refine our process for mapping. We are hoping for community feedback as we move forward so we can contribute high quality edits.
Are you going to Map other countries?
Yes. We are moving slowly to focus efforts on one country at a time to make sure our process is accepted by the OSM community out of respect for the process.
What is the Overall Process?
PHASE 1 - Generating Road Masks
  1. Training data is created by editors. Example
  2. Training data used by engineers in ML to produce road masks. Example
  3. Road masks are processed to remove low confidence predictions and add connections between short breaks.
PHASE 2 - Creating Road Vectors (.osm files)
1. Road masks are then processed using an algorithm from a black and white image to a vector format specifically a .osm file. Example
2. Our .osm file is then merged with current OSM changesets. During this process the following things take place
All current OSM data for an area is merged, keeping full history.
No changes are made to the current OSM data.
Duplicated roads are deleted from the ML generated data. Current OSM data is always taken over facebook generated data.
New ML identified roads are connected to current OSM roads.
3. We then have a locally stored file containing both our generated roads and the current OSM data set for a specific area.
PHASE 3 - Human Validation
1. We use the Tasking manager to divide up tasks and create a flow of editing and validation for the Facebook mapping team. Example
2. The Mapper picks a task and loads the locally stored .osm file generated from the post processing steps outlined above. Example
3. The task is opened up in our enhanced version of iD. (This includes both the roads generated by us and the current OSM data). Example
4. Our generated roads are highlighted in a different color so editors can inspect it for issues like crossing highways, disconnected roads, incorrect intersections, short road stubs, road types, etc., and fix these issues manually one by one, using Digital Globe satellite imagery background.
5. We do not delete current OSM data, but make typical OSM editing changes where necessary to ensure high quality of the data (more precisely follow imagery, consistent tagging, etc).
6. Our modified iD tool is equipped with data validation functionalities similar to JOSM and osmlint. This allows us to check for quality and conflicts with current OSM data. Some examples of what we check for include:
isolated roads that are not connected to rest of the map. Example-Before Example-After
roads crossing waterways.
roads with repeating node IDs.
roads with possible connections after a short extension. Example-Before Example-After
7. In case of conflicts between our newly created roads and other OSM editor's mapping, our mapper will almost always choose “keep their edits”. If we think our newly added roads are better aligned with the latest version of DG satellite imagery, we'll contact the other mapper offline to reach resolution. If we end up changing other mapper's edits, we will leave detailed notes to explain why.
8. Until all errors are fixed mappers will not be able to save. Here is an example of a highlighted issue prompting the mapper to fix it.
9. After fixing all the issues detected, our mapper clicks the “Save Local” button in the iD tool to save their edited roads locally for validation.
10. A second person, the validator then goes into the same task to verify that the data looks correct, makes necessary changes as needed and clicks “Submit” to finally upload the tile to OSM.

Controversy

References