CHSEGURA import

From OpenStreetMap Wiki
Jump to navigation Jump to search
CHSEGURA import - Other languages

About

The import process refers to hydrography dataset available on the administration of the Segura Hydrographic Confederacción. Segura basin is on the South-East quadrant of Spain

Segura Basin. Confederacción Hidrográfica del Segura.

Among the hydrographic data available on the Confederacción web, we have chosen only those that are useful and have equivalence with entities supported in OpenStreetMap, i.e. the layer of rivers, streams and canals (waterway=river and waterway=stream) and reservoirs layer landuse=reservoir.








Target

Our target is to add OpenStreetMap data supplied by this government agency which are not in OSM database yet. In case they exist in OSM but only partially (eg. rivers not completely mapped), merge current OSM data with data from Confederacción, giving precedence to what is currently in OSM over the new data.

The following images show a comparison between current data in OpenStreetMap (information obtained from Overpass_API, with those that will be added with the import. Only the main hydrographic network (rivers and streams) is compared and the improvement is noticeable.

River and stream. OSM.
River and stream. OSM+CH_Segura.




Animacion importacion.gif









Import Data

Links

Data Source site: C.H.Segura
Data Licence: Legal Notice
Type of license: CC BY 3.0
ODbL Compliance verified: Yes

It has been asked by email about permission to import the data, in order to ensure license compatibility with ODbl. The answer form Confederación Hidrográfica del Sagura is reproduced below: 


Received: by 10.59.8.229 with HTTP; Sun, 16 Feb 2014 23:30:50 -0800 (PST)
Date: Mon, 17 Feb 2014 08:30:50 +0100
Delivered-To: taborda.barroso@gmail.com
Message-ID: <CAGwCOQOVsjEn4ans3szMoYWEYa+UaSK6jOunxZQawWzfBko9Uw@mail.gmail.com>
Subject: Compatibilidad de licencia CC By con ODBL
From: =Matias_Taborda_Barroso?= <taborda.barroso@gmail.com>
To: segura@chsegura.es

Hola, buenos días.

Participo en un proyecto, desarrollado a través de Internet, llamado OpenStreepMap.org [1].

La idea es crear un mapa que esté disponible libremente y que pueda ser utilziado por todo el mundo,
al estilo de la Wikipedia.

Me gustaría preguntarles si podemos utilizar los datos cartográficos​ ​disponibles en su portal, en formato SHP, 
evidentemente haciendo caso a su Aviso Legal y siempre citando su origen, para incorporar esos datos al mapa de 
Openstreetmap. 

Por lo que puedo leer en su portal, sus datos estarían liberados bajo licencia CC By 3.0. En un principio, puede 
parecer que no debería haber ningún problema en la utilización de estos datos, pero la gente que trabaja en el 
mantenimiento de la base de datos de OSM es muy cuidadosa con el tema de las licencias de datos y me ha pedido 
que pregunte expresamente si se podrían utilizar esos datos y liberarlos bajo la licencia de Openstreetmap, 
es decir, una Licencia ODbL [2], lo que significa que ​:
​
(A) todo el mundo puede compartir (es decir, copiar, distribuir y usar para cualquier propósito) los datos,
(B) crear nuevos trabajos a partir de los datos, y
(C) adaptar, modificar, transformar y construir sobre esos datos, pero 

(A) indicando su procedencia, 
(B) liberando los datos derivados bajo esa misma licencia, y
(C) manteniendo los datos abiertos y accesibles para todo el mundo.

En otras palabras, se utilizarían sus datos junto con los ya existentes en un gran proyecto  ​conjunto para agregar 
valor a los datos.

La licencia ODbL es equivalente a la licencia Creative Commons Atribución-Compartir bajo la misma licencia
que usa el proyecto Wikipedia [3], sólo que es específica para datos.

Muchos de los datos que figuran en su mapa ya están incluidos en OSM, pero sin duda, con su colaboración, lograríamos 
enriquecer el mapa.


Atentamente,

 Matías Taborda.​


[1] http://www.openstreepmap.org
[2] http://opendatacommons.org/licenses/odbl/summary/
[3] http://creativecommons.org/licenses/by-sa/3.0/es/

 ----------------------------------------------------------------------

Disposition-Notification-To: "Vera Gomis Jose Antonio" <joseantonio.vera@chsegura.es>
Subject: RE: Compatibilidad de licencia CC By con ODBL
Date: Tue, 18 Feb 2014 14:17:20 +0100
From: "Vera Gomis Jose Antonio" <joseantonio.vera@chsegura.es>
To: <taborda.barroso@gmail.com>


Entiendo que por nuestra parte no debería haber ningún problema, pero no tengo claro que tenemos que hacer desde aquí.
Solo debemos indicarlo en nuestra web?. Hay que inscribirse en algún sitio?
 
Un saludo
 
################################################
José Antonio Vera Gomis
Jefe de Servicio de Sistemas de Información
Oficina de Planificación Hidrológica
Confederación Hidrográfica del Segura
Ministerio de Agricultura, Alimentación y Medio Ambiente 
Plaza de Fontes 1, 30001 Murcia
Tlf. +34 968 35 88 90 / +34 968 965 753 Fax +34 968 21 46 05
joseantonio.vera@chsegura.es http://www.chsegura.es <http://www.chsegura.es/>

OSM Data Files Example

  • Reservoir File. [1]

Import Type

Import will be done by humans (no bots) loading small files (up to 1.5 MB) in JOSM. Manual merging with already existing data will be done, giving priority to what is currently in OSM database and checking possible errors with the aid of aerial photography (Bing, PNOA).


Processing the Data

Data available at Confederación website are in SHP format and ED50 datum, Zone 30N (EPSG:23030), so it will be necessary to reproject the data and transform it to .osm format. Two layers will be used: Hydrographic network (Rivers and streams) and Reservoirs. The treatment will be similar for both of them, but we will separate their processing.

Hydrographic network

Hydrographic network shp file contains some values that need to be parsed and other ones that won't be used, as they are of no interest for OpenStreetMap.


Preprocessing

QGIS software has been used for preprocessing. Using the tools included in QGIS the following tasks have been achieved:

  • Fixing spelling errors: e.g.: Rio --- Río
  • Main name capitalization, according with Toponymy Guidelines from Instituto Geográfico Nacional. (e.g.: ARROYO DE LOS ENDRINALES ---- Arroyo de los Endrinales)
  • Adding a new field (TYPE), to be used later for tags assignation, differencing between rivers (waterway=river) and streams (waterway=stream).
  • Using the above field, two layers with streams and rivers entities are obtained separately. This step is also used for reprojection, saving the layers in datum EPSG: 4326.
  • A polygon layer from Confederación website is loaded, which divides the basin in 32 subareas following administrative criteria from the Hydrologic Plan of the basin. These subareas are not sub-basins, but they serve for the purpose. Subareas layer.
  • Intersection of lines hydrographic network layers (rivers and streams) and polygon subareas layer.
  • Multilayer division taking as split field the identifier of the subarea.


Tagging

Among the different fields in the original .shp file, only the name has been used, as numerical identifiers don't add any useful information. The field TYPE has been added to differentiate stationary waterways (streams) and permanent ones (rivers) and so have them ready for next step.

SHP ATTRIBUTE OSM TAG DESCRIPCTION
OBJECTID Not used. Numerical identifier used by C.H.Segura.
NOMBRE name=* Name of the waterway
SHAPE_len Not used. Entity lenght.
NIVEL Not used. Numerical category for internal CHS classification.
TIPO waterway=river, waterway=stream Field added with QGIS. Type of waterway.

Finally, according with import guidelines, the following tags about the import will be used in each Changeset:

Conversion to osm format

shp-to-osm.jar has been used to convert the files from previous steps to osm format. version 0.7.3 has been used, as the one on the wiki, 0.8.2, doesn't seem to work.

Rules file to convert dbf fields to osm tags is as follows: 

line,NOMBRE,,name,-
line,NOMBRE,N.C.,,
line,TIPO,Rio,waterway,river
line,TIPO,Arroyo,waterway,stream


Data simplification

Before importing the data, way simplification has been achieved, using GPSBABEL, with the following command: 

gpsbabel -i osm -f file.osm -x simplify,crosstrack,error=0.001k -o osm -F file-F.osm

Hydrographic network Import Process

Import will be done manually using JOSM. Process will be as follows:

  • Open *.osm file for a given area.
  • Depending on the file, there will be streams or rivers of different length. One of them will be selected and zoomed [3] on it.
  • OpenStreetMap imagery layer is activated. This way a quick check is easily done to see if the given waterway is already in OSM database, total or partially.
  • If it is partially in OSM, the area is downloaded from OSM in a separate layer. The object is then selected in the osm file to be imported and merged with downloaded layer (Ctrl+Shift+M) and deleted from the original file.
  • It is then checked if the way is correctly traced, using aerial imagery (PNOA, Bing).
  • Ways downloaded from OSM and obtained from the import are connected to complete the waterway.
  • Process will be similar in cases where a given waterway is not in OSM but the main waterway it is connected to is already in OSM database. Corresponding nodes will have to be merged.
  • Changeset will be tagged as described above.
  • Finally data is uploaded to OSM.

Postprocessing

Once data is imported, a check is done to detect places where the waterway crossed other ways: roads, tracks, etc. Bridges/tunnels are created where necessary and uploaded to OSM with a regular user account.

Reservoirs

The available reservoir layer, as the one of rivers, is in shp format. It is a polygon layer, representing a total amount of 36 reservoirs of the basin.


Preprocessing

QGIS has been used for preprocessing in the same way than for hydrographic network layer. Using QGIS Field calculator the following edits have been done:

  • Spelling normalization of the field containing reservoir names. (e.g.: EMBALSE DE LA RISCA ----> Embalse de la Risca).
  • Reproject the layer to EPSG:4326.


Tagging

Only the field containing reservoir names has been used, as the rest are considered useless for OpenStreetMap.

SHP ATTRIBUTE OSM TAG DESCRIPTION
OBJECTID Not used Object numerical identifier.
NOMBRE Not used It contains the name, but field NOMBRE_COM is more complete.
ID_CHS Not used Internal C.H.S. numerical identifier.
NOMBRE_COM name=* Full name of the reservoir.
SHAPE_Leng Not used Reservoir boundary length.
SHAPE_Area Not used Area occupied by the reservoir surface.
USO Not used Use of the water contained in the reservoir.

According to guidelines from discusión the following tags are added:

Changesets are tagged as follows :

Conversion to osm format

Conversion to osm format is done with version 0.7.3 of Shp-to-osm.jar.

Rules file to convert dbf fields to osm tags is as follows: 

outer,NOMBRE_COM,,name,-
outer,NOMBRE_COM,,natural,water
outer,NOMBRE_COM,,water,reservoir


Reservoir Import Process

Import process for reservoirs will be similar to the one described for hydrographic network, but as the number of entities is smaller (36) it is not necessary to split the file and the work will be done directly on the file obtained in previous transformation.

  • Open .osm file with JOSM.
  • Openstreetmap imagery layer is loaded.
  • A reservoir is selected and zoomed on it [3].
  • It is checked whether the reservoir is already in OSM or not, using OpenStreetMap imagery layer. If it is in OSM database, then is removed from the .osm file and the next object is processed.
  • If it doesn't exist in OSM, the area is downloaded in a new layer and the reservoir is merged with it (Ctrl+Shift+M). Nodes of the waterway that feeds the reservoir are connected where appropriate. Note that if hydrographic network has been previously imported, all reservoirs will have their corresponding waterway.
  • The quality of data is assessed activating some aerial photography layer (PNOA, Bing)
  • Finally data is uploaded. Import tags are added to the changeset.
  • Above steps are repeated for the next reservoir.


User account

import_chsegura user account, will be used for the import.

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=CHSEGURA_import&oldid=1607669"