Import/Catalogue/ItalyFuelStations/maintenance

About

This page is about keeping updated fuel stations imported from Ministero dello Sviluppo Economico (MISE) dataset. Offical dataset name is Prezzi praticati e anagrafica degli impianti.

Dataset will be conflated in order to generate OSM files suitable to be uploaded in planet.osm. It will not be a complete blind import: data resulting from conflation will be audited on province based areas by mappers through audit support maps; province extension has been chosen because of its limited POI number (ie: 100-300 nodes).

These updates are being discussed on the national OSM mailing list. This wiki page is the result of consensus there.

Goals

MISE source is issuing daily updates (defined herein as "Quotidianamente vengono pubblicate in questa sezione le informazioni in vigore alle ore 8 del giorno precedente a quello di pubblicazione."); hence a periodic update can be performed.

Tools

The following tools are needed to accomplish maintenance:

• Datasets download and pruning script: run.sh
• Data cleanup and transformation: OpenRefine
• Integration with OSM: OSM conflator

Schedule

First import has been performed in mid 2018 after community audit trackable in project page.

A pilot maintenance is ready to be uploaded. Because of its limited size, Udine province has been chosen.

Import Data

Datasets are presented in csv format. Script run.sh will download files and make some filter and simplifications. The following example will download daily CSVs (if not present) and keep only Gorizia (GO) province; multiple provincies allowed in arguments. $ ./run.sh GO

Record format

Please refer to import page

Legal

Please refer to import page

Import Type

Dataset will be imported on a province base. After conflation, audit link will be published in table Elenco delle provincie and an audit announcement will be made in talk-it ML and OSM Telegram channel. Audit process allows shared check for inconsistencies and minor problems: fixme tag can be added/edited if needed.

Data Preparation

The data is presented as two csv files. "Prezzo alle 8" file will be cross referenced in "Anagrafica" file. Prior to conflation, the following operations will be applied:

Anagrafica

• void values removal
• type conversions string>number
• quotes removal
• title case and title case adjustments

For the full list, refer to anagrafica.operations file.

Prezzo alle 8

• fuel type mapping
• survey date filtering
• duplicate removal

Fuel station operators are requested by MISE to update their record weekly, hence date filtering has been set to 2 months. For the full operations list, refer to prezzo_alle_8.operations file.

Tagging Plan

Fuel type tagging will based on Octane Rating wiki. The following tags will be set accordingly by cross-referencing prezzo_alle_8 and anagrafica:

• fuel:lpg
• fuel:cng
• fuel:octane_95
• fuel:octane_98
• fuel:octane_100
• fuel:octane_101
• fuel:diesel

Conflation

Fuel stations already in OSM will be extracted using the overpass-turbo queries. Udine province query can be used as a template.

Conflation is performed by OSM Conflator. Existing OpenStreetMap data within a range is merged and tags will be replaced/updated accordingly to conflator parameter file.

Workflow

pi@raspberrypi:~/OSM $ conflate -i UD.json --osm <[http://overpass-turbo.eu/s/Gf6 queried data] raw export>  -v -c preview_UD.json ../profile-mise.py

Upload preview_UD.json in audit tool Wait for audit to be completed. Then run again with audited data:

pi@raspberrypi:~/OSM conflate -i UD.json -a audit_FuelUD.json  -o readytoimportUD.osm ../profile-mise.py

Once completed, resulting file fixme's will be cleared (typos, minor position corrections) and osm file will be published in maintenance wiki page (IT).

Dedicated upload account

The account attilaimport will be used to upload community revised .osm files.

Changeset Tags

Changeset will be tagged with:

• source=MISE - Ministero Sviluppo Economico
• source:date=yyyy-mm-dd

Team Approach

Import will be managed by the following OSM users:

• Cascafico

QA and TODO

In case of import problem changeset involved will be reverted using proper reverter. Non-matching POIs are tagged by fixme=This object might have been dismantled, please check.