Mexico Road Name Import Project
- 1 About
- 2 Import Plan Outline
- 3 Goals
- 4 Schedule
- 5 Progress
- 6 Import Data
- 7 Data Preparation
- 8 Data Merge Workflow
- 9 QA
- 10 See also
Kaart will conflate data provided by Mexico’s National Institute of Statistics and Geography (INEGI) with OSM. The team dedicated to this import will use Cygnus to conflate the Marco_Geoestadístico_Nacional_(MGN) data set.
Kaart is following the import guidelines and throughout the duration of the project will continue efforts to ensure effective communication with Mexico’s OSM community.
Import Plan Outline
- Determine which INEGI data set is most effective for an import → MGN (June 2018)
- Develop tools to efficiently and accurately convert .shp files to .osm.pbf and accurately map attributes
- Develop processes to edit and validate data using JOSM (including custom paint styles and validators
- Conflate MGN data with OSM data using Cygnus one state at a time
- Create projects for each state using an instance of the OSM_Tasking_Manager
To contribute up-to-date data for the road networks of Mexico by updating missing geometry and road names.
- Discussion with Mexico's OSM community - September 2018
- Announcement to Imports mailing list - September 2018
- Develop data processing tools - September 2018
- MGN data conflation - October 2018 → Completion(TBD)
- Manual editing and validation using JOSM - October 2018 → Completion(TBD)
- Started Mapping: October 2018
- Finished Mapping: December 4, 2018
- Started Mapping: December 4, 2018
- Started Mapping: December 12, 2018
- Finished Mapping: January 8, 2019
The data is delivered as a .zip (~2.6GB) of the entire country of Mexico which contains a .zip for each state. Each state has a corresponding number(id) prefixing the name of the state (ex. 01_aguascalientes) as shown in the PDF on the right.
The state .zip contains 3 directories:
|catalogos||PDF and .txt files describing the data sets|
|conjunto de datos||"Data Sets" .shp files|
There are 13 different data sets within conjunto de datos for each state (information describing them can be found in contenido.pdf in catalogos).
The data set containing road data is Ejes de vialidad which will be identified by the id prefixing the letter "e" (ex. 01e.shp).
OSM Data Files
OSM data will be processed and delivered on a state by state basis. If there are any questions or concerns regarding the OSM data, please contact User:Zlavergne (examples of data may be made available).
This is a one-time import using a combination of automated tools for data processing/conflation and manual editing/validation prior to upload.
- Automated tool to prepare data for conflation
- Automated tool to conflate data and output OSM_XML files
- Progress tracking will be done using an instance of the OSM_Tasking_Manager
- Manual processes for editing and validating data in JOSM
Data Reduction & Simplification
Our import focuses on road names and missing road geometries (we will also add oneway information to missing geometries). As described in the Import Data #Background section, we will be using the Ejes de vialidad data set.
In order to use the Cygnus conflation tool, the .shp files must first be converted to .osm.pbf files that are <20MB and no larger than 50km x 50km areas.
Our team developed a tool to automate the process of splitting and converting. Simply explained below:
- Translation file (compatible with ogr2osm)
- Size of area (in meters)
- Creates a grid of squares covering extent of Shapefile
- Executes spatial analysis to extract Shapefile data within each grid into individual Shapefiles
- Converts Shapefile to .osm and translates attributes to OSM tags
- Converts .osm file to .osm.pbf
We then run each .osm.pbf file through Cygnus twice:
- Once with name as the tag of interest without missing geometry
- Once to generate missing geometry without any tags of interest
We will end up with 2 OSM_XML files for each square. We will use JOSM to merge the name files for all squares of a state into one OSM_XML and missing geometry into another.
Once we have two conflated OSM_XML per state (one for names, one for missing geometry), we can then create an OSM_Tasking_Manager project for the state and use the task bounds to take the appropriate Bounding_box of the conflated files as separate layers into JOSM.
All of these processes have been extensively tested and their outputs will be closely monitored throughout the import process.
|MNG Field||OSM Tag||Description (Sp)||Description (En)|
|NOMVIAL||name||NOMBRE DE VIALIDAD||Name of Road|
|TIPOVIAL||highway||TIPO DE VIALIDAD||Type of Road|
|SENTIDO||oneway||SENTIDO DE LA VIALIDAD||Sense of the Road|
|CVE_SEG||CLAVE DEL SEGMENTO DE VIALIDAD CONFORME A LAS MANZANAS||Key of road segment according to blocks|
|CVE_VIAL||inegi:road_key||CLAVE DE LA VIALIDAD||Road Key|
|CVE_LOC||CLAVE DE LOCALIDAD||Locality Key|
|CVE_MUN||CLAVE DE MUNICIPIO||Municipality Key|
|CVE_ENT||CLAVE DE ENTIDAD FEDERATIVA||Federal Entity Key|
We will use the following changeset comments: Importé geometría faltante utilizando datos de INEGI. #Kaart for the missing geometry and Actualicé nombres de vías utilizando datos de INEGI. #Kaart for the road name validation. We will use the changeset source: source=INEGI:MGN:06-2018
Data Transformation Results
OSM XML files will be created on a state by state basis to keep the data as accurate as possible.
Data Merge Workflow
The development of the tools and the actual import of the data will be handled by the Kaart Team with constant collaboration with the local Mexico community.
Once the user selects a tasks they will follow the following workflows:
- Download OSM data within task area using an Overpass query
- Merge the missing-geometry layer with the OSM data layer
- Manually verify connections
- Run JOSM/Validator with the custom validator
- Upload changeset using the comment: Mexico Road Geometry Import #Kaart
- Use the name layer to manually verify road names
- Run JOSM/Validator with the custom validator
- Upload changeset using the comment: Mexico Road Name Project #Kaart
Any necessary reverting will be handled with JOSM Reverter Plugin.
Conflation will be handled by Cygnus and manually verified by our team.
You can view changesets using OSMCha.
The email to the Imports mailing list was sent on 2018-09-18 and can be found in the archives of the mailing list at .
The email to the Talk-mx mailing list was sent on 2018-10-2 and can be found in the archives of the mailing list at .