Madrid Places of Worship Import

From OpenStreetMap Wiki
Jump to: navigation, search
Available languages — Madrid Places of Worship Import
· Afrikaans · Alemannisch · aragonés · asturianu · azərbaycanca · Bahasa Indonesia · Bahasa Melayu · Bân-lâm-gú · Basa Jawa · Basa Sunda · Baso Minangkabau · bosanski · brezhoneg · català · čeština · corsu · dansk · Deutsch · eesti · English · español · Esperanto · estremeñu · euskara · français · Frysk · Gaeilge · Gàidhlig · galego · Hausa · hrvatski · Igbo · interlingua · Interlingue · isiXhosa · isiZulu · íslenska · italiano · Kiswahili · Kreyòl ayisyen · kréyòl gwadloupéyen · Kurdî · latviešu · Lëtzebuergesch · lietuvių · magyar · Malagasy · Malti · Nederlands · Nedersaksies · norsk bokmål · norsk nynorsk · occitan · Oromoo · oʻzbekcha/ўзбекча · Plattdüütsch · polski · português · română · shqip · slovenčina · slovenščina · Soomaaliga · suomi · svenska · Tiếng Việt · Türkçe · Vahcuengh · vèneto · Wolof · Yorùbá · Zazaki · српски / srpski · беларуская · български · қазақша · македонски · монгол · русский · тоҷикӣ · українська · Ελληνικά · Հայերեն · ქართული · नेपाली · मराठी · हिन्दी · भोजपुरी · অসমীয়া · বাংলা · ਪੰਜਾਬੀ · ગુજરાતી · ଓଡ଼ିଆ · தமிழ் · తెలుగు · ಕನ್ನಡ · മലയാളം · සිංහල · བོད་ཡིག · ไทย · မြန်မာဘာသာ · ລາວ · ភាសាខ្មែរ · ⵜⴰⵎⴰⵣⵉⵖⵜ ⵜⴰⵏⴰⵡⴰⵢⵜ‎ · አማርኛ · 한국어 · 日本語 · 中文(简体)‎ · 中文(繁體)‎ · 吴语 · 粵語 · ייִדיש · עברית · اردو · العربية · پښتو · سنڌي · فارسی · ދިވެހިބަސް

Goals

The goal is to manually merge and import places of worship from the Madrid city council database, see: Import_Ayuntamiento_Madrid (in spanish)

In Madrid there are 180 places of worship already in OSM. The dataset from the city council contains 498. It contains useful information like: name, description, opening hours, if it is accesible with a wheelchair or not, address, phone, email and url.

Existing hand mapped information will not be replaced by the import without on-the-ground verification.

Schedule

  1. October 24 2016: We started the preparation and the discussion. See the discussion in the talk-es list (in spanish)
  2. October 30: We started the import
  3. November ?: We finish doing the QA after the import
  4. November ?: Notify talk-es and the City Council with a report

Import Data

Data description

We'll use the xml files for processing. You can download the data here and here for the non-catholic.

Background

ODbL Compliance verified: YES

The license can be consulted here (in Spanish). It requires attribution of the source, so all nodes and the changeset will include source=Ayuntamiento de Madrid. The Madrid City Council (Ayuntamiento de Madrid) is already on the contributors list.

After sending the report with the problems and errors found during the drinking water import, including a section about the license, got a response from the "Subdirección General de Transparencia del Ayuntamiento de Madrid" (the data owner), and they confirmed me that we comply with their attribution clause using their data in OSM if we clearly specify the source and the source:date, the way it was done on the Madrid_Drinking_Water_Import. And they do like the work we are doing :) Kresp0 (talk) 13:30, 16 October 2016 (UTC)

Compatibility with the ODbL was already discussed in the imports and talk-es mailing lists.

Cumplimiento de la cláusula de atribución de los datos abiertos del Ayto. de Madrid en OSM.pdf

Import Type

The import will be done manually, using the Tasks Manager, so it can be assigned to different volunteers. JOSM will be used to edit the data.

Data Preparation

Data Reduction & Simplification

Some corrections are made with the data conversion script madridxml2osm and some tags added and removed manually using JOSM. See: "Data Transformation". Some of the fields aren't relevant and will be ignored.

Tagging Plans

Here are the original fields, their meaning and how they will be converted to the resulting OSM file:

Tagging Conversion
Original Fields Meaning OSM tag Comments
ID-ENTIDAD Internal reference number Not used
NOMBRE Name name=*
DESCRIPCION-ENTIDAD Description description=* Part of description=*
HORARIO Timetables opening_hours=* Manual processing needed
EQUIPAMIENTO Additional info about the place description=* Part of description=*
TRANSPORTE Nearest bus and metro stations Not used
DESCRIPCION Description description=* Part of description=*
ACCESIBILIDAD Wheelchair access. 0 = no access, 1 = some kind of access wheelchair=yes or wheelchair=no Not used, because all are set to 0, except one.
CONTENT-URL URL url=* Only if JOSM likes the url
NOMBRE-VIA Name of the street, without the prefix. addr:street=* Part of addr:street=*
CLASE-VIAL Street prefix addr:street=* Part of addr:street=*
TIPO-NUM kind of housenumber Not used
NUM housenumber addr:housenumber=*
PLANTA Floor Not used
PUERTA Door Not used
ORIENTACION Orientation Not used
LOCALIDAD City Not used
PROVINCIA Province Not used
CODIGO-POSTAL Post code addr:postcode=*
BARRIO District Not used
DISTRITO District Not used
COORDENADA-X Coordinate X Not used
COORDENADA-Y Coordinate Y Not used
LATITUD Latitude OSM Latitude
LONGITUD Longitude OSM Longitude
TELEFONO phone phone=*
FAX Fax fax=*
EMAIL Email email=*
TIPO Kind Not used

We added the following tags:

Changeset Tags

We will use the following changeset tags:

Data Transformation

Data is in xml format. We already:

  1. Converted it to OSM format using the madridxml2osm data conversion script:
madridxml2osm.sh 209426-0-templos-catolicas.xml amenity=place_of_worship
  1. Edited Templos_e_iglesias_católicas.osm using JOSM and added this tags
  • religion=christian
  • denomination=roman_catholic
  • building=church (those with "Catedral" in the name got building=cathedral)
  1. Non-catholic temples processing:
madridxml2osm.sh 209434-0-templos-otros.xml amenity=place_of_worship


  1. Added most of the non-catholic places religion and denomination based on the name.
Nombre religion denomination
Iglesia Anglicana Episcopal de San Jorge christian anglican
Iglesia Bautista Alameda de Osuna christian baptist
Iglesia Bautista Barrio del Pilar christian baptist
Iglesia Bautista de Usera christian baptist
Iglesia Bautista de Vallecas christian baptist
Iglesia Bautista El Buen Pastor christian baptist
Iglesia Bautista General Lacy christian baptist
Iglesia Bautista Pueblo Nuevo christian baptist
Asociación Betel christian evangelical
Centro de Evangelización Anuncio de Paz christian evangelical
Iglesia Cristiana Evangélica de Aluche christian evangelical
Iglesia Cristiana Evangélica de Chamartín christian evangelical
Iglesia Cristiana Evangélica en Hortaleza (Eben Ezer) christian evangelical
Iglesia Evangélica Adulam de Madrid christian evangelical
Iglesia Evangélica Casa de Oración christian evangelical
Iglesia Evangélica Casa de Oración de Madrid christian evangelical
Iglesia Evangélica China Española christian evangelical
Iglesia Evangélica Cristo Vive christian evangelical
Iglesia Evangélica de Hermanos El Shadday christian evangelical
Iglesia Evangélica de Hermanos Resurrección y Vida christian evangelical
Iglesia Evangélica del Nazareno christian evangelical
Iglesia Evangélica Eben Ezer Dios es amor christian evangelical
Iglesia Evangélica Encuetro con Dios christian evangelical
Iglesia Evangélica Española El Salvador christian evangelical
Iglesia Evangélica Fuente de la Vida christian evangelical
Iglesia Evangélica Nueva Vida christian evangelical
Iglesia Evangélica Pentecostal Amistad Cristiana christian evangelical
Iglesia Evangélica Pentecostal Bethel christian evangelical
Iglesia Evangélica Pentecostal Salem christian evangelical
Iglesia Evangélica Pueblo Nuevo christian evangelical
Testigos Cristianos de Jehová. Avda. Moratalaz christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Andaluces christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Arroyo christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Artistas christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Carrascales christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Chantada christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Doctor Gómez Ulla christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Guamo christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Hilario Sangrador christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Ignacio Ellacuría christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Jacobinia christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Las Pedroñeras christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Luis Ruiz christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Marcudos christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Martín de Vargas christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Mineros christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Pérez Herrera christian jehovahs_witness
Testigos Cristianos de Jehová. Calle San Moisés christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Santa Ana Baja christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Sierra del Sabiñar christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Sierra Salvada christian jehovahs_witness
Testigos Cristianos de Jehová. Calle Tembleque christian jehovahs_witness
Iglesia Cristiana Adventista del Séptimo Día. Madrid Alenza christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Aluche christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Beneficencia christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Calatrava christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Eben-Ezer christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Efes christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Emaús christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Grupo brasileño christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Noviciado de Horeb christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Resurrección christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Vallecas christian seventh_day_adventist
Iglesia Cristiana Adventista del Séptimo Día. Madrid Ventas christian seventh_day_adventist
Asamblea Cristiana de Madrid christian
Asociación Darse - Dar Ayuda para la Reinserción Social España NO?
Catedral Anglicana de Madrid christian
Catedral Ortodoxa Griega christian greek_orthodox
Centro Cultural Islámico. Mezquita de Madrid muslim
Comunidad Cristiana Maranatha christian
Iglesia Celebración Cristiana christian
Iglesia CENFOL España christian
Iglesia Centro Cristiano Adonay christian
Iglesia Comunidad Cristiana de Fe christian
Iglesia Cristiana Hosanna christian
Iglesia Cristiana Reformada christian
Iglesia Cristiana Shalom Adonai christian
Iglesia Cristiana Verbo christian
Iglesia Cuerpo de Cristo, Don Pedro christian
Iglesia Cuerpo de Cristo, General Ricardos christian
Iglesia de Cristo IEE christian
Iglesia de Cristo - Madrid christian
Iglesia de Dios (Verbo de Dios) christian
Iglesia de Filadelfia Orcasur christian
Iglesia de Jesús christian
Iglesia de la Cienciología scientologist
Iglesia de Los Elegidos de Dios
Iglesia de los Hermanos en La Vaguada
Iglesia Dios es Amor - Unión Cristiana
Iglesia Ejército de Salvación, Hermosilla
Iglesia El Buen Pastor de Madrid
Iglesia Emanuel (U.C.E) Orcasitas (Orcasur)
Iglesia Española Reformada Episcopal
Iglesia Evangelio Cuadrangular Vida Nueva Usera christian
Iglesia Filadelfia del Centro Cielos Abiertos christian
Iglesia Internacional Alianza de Amor
Iglesia Misionera Asamblea Cristiana christian
Iglesia Resurrección IEE
Iglesia Servidores de Vida
Iglesia Tabernáculo de Alabanza
Iglesia Templo de Dios
Mezquita Abu Bakr muslim
  1. This code adds religion and denomination to non-catholic temples. It reads the file "traduccion nombre a religion y denomination.csv" and modify Templos_e_iglesias_no_católicas.osm
while IFS='' read -r line || [[ -n "$line" ]]; do

    NOMBRE=`echo $line | awk -F ';' '{print $1}'`
    RELIGION=`echo $line | awk -F ';' '{print $2}'`
    DENOMINATION=`echo $line | awk -F ';' '{print $3}'`

    if [ -z "$DENOMINATION" ]; then
        if [ -z "$RELIGION" ]; then
           echo $NOMBRE SIN RELIGION NI DENOMINATION
        else
           echo 's/k="name" v="'$NOMBRE'"\/\>/k="name" v="'$NOMBRE'"\/\>\<tag k="religion" v="'$RELIGION'"\/\>/g Templos_e_iglesias_no_católicas.osm' | perl -pe "s/g Templos_/g' Templos_/" | perl -pe "s/s\/k/perl -pe 's\/k/g" > /tmp/l
        sh /tmp/l > r ; mv r Templos_e_iglesias_no_católicas.osm 
        fi
    else
        echo 's/k="name" v="'$NOMBRE'"\/\>/k="name" v="'$NOMBRE'"\/\>\<tag k="religion" v="'$RELIGION'"\/\>\<tag k="denomination" v="'$DENOMINATION'"\/\>/g Templos_e_iglesias_no_católicas.osm' | perl -pe "s/g Templos_/g' Templos_/" | perl -pe "s/s\/k/perl -pe 's\/k/g" > /tmp/l
        sh /tmp/l > r ; mv r Templos_e_iglesias_no_católicas.osm
    fi
    
    done < "traduccion nombre a religion y denomination.csv"
  1. Merged the layers
  2. Added the tag:
  • source:date=2013-05-01
  1. Removed the tags:
  • addr:district
  • addr:suburb
  • addr:province
  • addr:city
  • wheelchair
  1. Saved as Lugares_de_culto_de_Madrid.osm
  2. Manually drawn the working area using JOSM and saved as geojson
  3. Uploaded the working area to the TM
  4. Splitted the osm file to have one file for each task on the TM using the osmboxes script

Data Import Workflow

Team Approach

Import will be undertaken by experienced OSM volunteers, following a strict workflow.

References

The import will be discussed in the Talk-Es list.

Workflow

As most (if not all) of the volunteers will be Spanish speaking, they can follow the detailed workflow in Spanish language on the TM.

The workflow will be as follows:

  • 1. Install the TODO list JOSM plugin if the volunteer don't have it.
  • 2. Create, if we don't have one yet, an import specific user account, like username_imports (you will need a different email address from the account you used for your regular OSM account). Change the OSM username to that specific account in the JOSM preferences.
  • 3. Configure JOSM to allow remote access.
  • 4. Login into the Tasks Manager.
  • 5 Select one square to work on.
  • 6 Open the OSM data and the temples data by clicking on the JOSM button and the link on the extra instructions.
  • 7 Select all nodes on the temples layer and add them to the ToDo list. For each node we'll:
    • Edit the description=* to be less than 256 characters
    • On the addr:street=*, fix the name to match the current OSM street name
    • opening_hours=* has to be translated from spanish to opening_hours=* format
    • Remove building=church if the building where the place of worship is located is not a church.
    • Check the rest of the tags
    • Conflate the node data with the temple building/node if there is already one or create a new one if not. Don't replace existing hand mapped information without on-the-ground verification.
  • 8. After the import is done, there will be a review process using the TM.

Updates

Each year I'll download the dataset, do a diff and update the data to OSM manually. Until I get bored. Kresp0 (talk) 17:41, 24 October 2016 (UTC)

Reverse plan

In case of any trouble, JOSM reverter will be used.