Madrid Bicycle Parking Import

From OpenStreetMap Wiki
Jump to navigation Jump to search
Madrid Bicycle Parking Import - Other languages

Goals

The goal is to import bike parkings from the Madrid city council database, see: Import_Ayuntamiento_Madrid (in spanish)

In Madrid there are 180 bike parkings already in OSM. We'll merge them.

Schedule

  1. Preparation, discussion - due to start the 1st of April 2016
  2. March 29: sent mail to talk-es: confirmed license compatibility and data usefulness for OSM
  3. April 1: sent mail to talk-es with the link to this wiki page: no objections made.
  4. April 3: sent mail to Imports
  5. Import - expected to start any time after the community has solved any issues, doubts or concerns about this import. In progress.
  6. April 10 - Import finished. Sent mail to talk-es

Import Data

Data description

The original dataset is shp format. It includes a total of 1,185 spots. Some of them are reported stolen or broken and therefore won't be imported.

We'll use the shp file for processing. You can download the data here.

Background

ODbL Compliance verified: YES

The license can be consulted here (in Spanish). It requires attribution of the source, so all nodes and the changeset will include source=Ayuntamiento de Madrid. The Madrid City Council (Ayuntamiento de Madrid) is already on the contributors list.

After sending the report with the problems and errors found during the drinking water import, including a section about the license, got a response from the "Subdirección General de Transparencia del Ayuntamiento de Madrid" (the data owner), and they confirmed me that we comply with their attribution clause using their data in OSM if we clearly specify the source and the source:date, the way it was done on the Madrid_Drinking_Water_Import. And they do like the work we are doing :) Kresp0 (talk) 13:20, 16 October 2016 (UTC)

Compatibility with the ODbL was already discussed in the imports and talk-es mailing lists.

Cumplimiento de la cláusula de atribución de los datos abiertos del Ayto. de Madrid en OSM.pdf

All nodes and the changeset will include source=Ayuntamiento de Madrid / XXXX, where XXXX is a department or organization in the tagging. Each node has also a source for the photo, that will be included too. Because the source for each node and for each photo could be various departments and organizations, we have to respect that and use the source=* for each node.

Import Type

The import will be done manually, with the data divided in sets of 50 nodes, so it can be assigned to different volunteers.

Data Preparation

Data Reduction & Simplification

As mentioned before, the data file we have chosen is in shp format. Some corrections are part of the data conversion script. Some of the fields aren't relevant and will be ignored, while some others have been used to discard the node.

Tagging Plans

Here are the original fields, their meaning and how they will be converted to the resulting OSM file:

Tagging Conversion
Original Fields Meaning OSM tag Comments
OBJECTID Internal reference number Not used
CODIGO Internal reference number Not used
CENTRO Name of the location where the parking is. It will be appended to the name=* tag, after the "Aparcabicis" word.
DIRECCION Street, number. Not used. Sometimes contains only the street. Would be addr:street=* and addr:housenumber=*.
DISTRITO District code Not used.
TX_DISTRIT District name Not used.
ARCOS Number of stands Not used.
CAPACIDAD Number of bikes that can be parked capacity=*
MODELO Parking model:

1: stands

2: bollard

3: anchors

5: wall_loops

6: stands

7: mixed

8: mixed

bicycle_parking=*
TITULAR Operator operator=* Most operated by the city council, some by the UAM, 1 by a particular and some have this field empty.
ESTADO If this field is not empty, means that there is some problem with the bike parking, explained in OBSERVACIO.
OBSERVACIO Description of the problem. 147 entries have problems like missing stands or parking signals. If reported stolen or broken won't be imported. description=*
FOTO Filename with a picture, like "PBI_01021.jpg" It will be appended to the image=*, after "http://www.infobicimadrid.es/fotos/aparcabicis/small/"
FECHA_FOTO When the picture was taken Not used.
FUENTE_FOT Source for the picture. source:image=*
FUENTE Source. We'll append it to source=*, after "Ayuntamiento de Madrid /" Several sources, most from city council deparments.
ID_INFORMO Internal reference number Not used
ID_OBRAS ? Not used
CODIGO_GEN ? Not used
ELIMINADO Removed. Used to discard the node 93 of them have non zero values, seems that those are no more.
enplanos "in maps". Not used. Seems that some "removed" are still shown in maps.

To all the nodes, we will add the following tag:

Changeset Tags

We will use the following changeset tags:

Data Transformation

Data is in shp format. After converting the file to osm format using JOSM with the opendata plugin, we process the file using a shell script and divide it in chunks of 50 nodes for importing using xml_split.

Data Import Workflow

Team Approach

Import will be undertaken by experienced OSM volunteers, following a strict workflow. Each volunteer will choose a file with 50 nodes and will import them to OSM, while merging them with the old ones they may encounter.

References

The import will be discussed in the Talk-Es list and in the Imports list.

Workflow

As most (if not all) of the volunteers will be Spanish speaking, they can follow this detailed workflow in Spanish language, with screenshots.

The workflow will be as follows:

  • 1. Install the TODO list JOSM plugin if the volunteer don't have it.
  • 2. Create, if we don't have one yet, an import specific user account, like username_import_aparcabicis_madrid (you will need a different email account from the account you used for your regular OSM account). Change the OSM username to that specific account in the JOSM preferences.
  • 3. Download one of the files with a subset of 50 nodes that hasn't been imported yet, and write your username and mark that subset as Work in progress in the table of Madrid_Bicycle_Parking_Import_Progress. Open that file in JOSM and another file from overpass turbo with the OSM data.
  • 4. Activate the layer with the bike parking to review.
  • 5 We go through all the 50 nodes, one by one, using the ToDo list. For each node we first check its correctness, correcting any errors or typos we may still encounter. If the node is clearly wrong or suspicious of being wrong, it won't be imported in the first place, and it will be added to the comments in the table of Madrid_Bicycle_Parking_Import_Progress, so it can be checked afterwards by other mappers to take a decision about it. The mapper will delete that node to make sure it won't be imported.
  • 6. After checking the correctness of the node, we will proceed with the conflation of this new node with the old nodes, in case it is duplicated. To do so, we delete the tags that aren't needed or wrong, we change or correct those tags that need so and finally we select both the new and old nodes and we merge them with the Merge tool (M), so the old node moves to the new node location, keeping the id and history of the old node, and getting the new node tags added. Keep in mind that the conflation has to be done following the Conflation rules.

Reverse plan

In case of any trouble, JOSM reverter will be used.

Conflation

There are 180 bike parkings in the OSM database for the Madrid area. Some of them are independent of the new ones, and therefore we will leave them in the map unchanged. But many others are a duplicate of one of the new nodes. In this last case we will use the JOSM Merge tool (M), with which we will keep the node id of the old one (and therefore its history), transfer any useful data from the old one (like covered=* for example) and change its location to the new one. As the M tool merges all data, before doing so we will manually delete any tag that isn't needed or it's wrong in the old node, like addr:country=*.

As explained before, if in doubt don't do anything with the node and report the problem in the comments field in the table of the Import Progress wiki.

Updates

Each 6 months I'll download the dataset, do a diff and update the data to OSM manually. Until I get bored. Kresp0 (talk) 07:51, 24 July 2016 (UTC)

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Madrid_Bicycle_Parking_Import&oldid=1417730"