JOSM/Plugins/OpenData

From OpenStreetMap Wiki
< JOSM‎ | Plugins
Jump to: navigation, search

Introduction

The JOSM plugin Open Data is designed to ease analysis of geographic data obtained from various open data portals providing data compatible with OSM. It supports a variety of file formats, allows to apply well known OSM tags to specific data (via corresponding modules performing an appropriate field mapping), download existing data in OpenStreetMap (via Overpass API), and will allow to perform a graphical "diff" between them (not yet implemented. It is likely to be performed via Conflation plugin).

Installation

To get the last version, you need to have JOSM 5132 installed. Open JOSM and select from menu Edit > Preferences, then there Plugins > Download list. Check opendata in the list. Now restart JOSM. Note that loading in large files consumes memory and you may wish to restart JOSM with more memory than you have previously. If you find JOSM pauses and is slow when importing a file, this may be due to a memory shortage rather than the plugin itself.

Core plugin features

The plugin allows basically to do two things: import data, download corresponding OSM data.

File formats

Installing the plugin allows JOSM to read these files formats:

File format Extension(s) Implementation Comments
CSV *.csv custom
OpenDocument *.ods jOpenDocument File format used by OpenOffice.Org and LibreOffice
Excel *.xls Apache POI File format used by Microsoft Office
KML *.kml, *.kmz custom File format used by Google Earth
GML *.gml custom
MapInfo Interchange *.mif custom
MapInfo TAB *.tab custom + GeoTools
ESRI Shapefile *.shp GeoTools See also Shapefiles.
Zip *.zip java.util
7z *.7z J7Zip
NEPTUNE *.xml custom NEPTUNE is an exchange format used in France to describe public transport information.

You do not need to install any module to open these files. You can open files just by drag-and-drop then in main JOSM window, or by using File->Open and select the corresponding file filter. However, the plugin behaviour may differ, depending on the file format, see below. If you have a file that cannot be opened with the plugin, let me know on the talk page, or better, create a new ticket on the JOSM bugtracker (component plugin opendata). Do not forget to attach or link to your file if you want the feature to be implemented.

Tabular files (CSV, XLS, ODS)

Tabular and spreadsheet files are a convenient way to exchange textual data, but are not an ideal solution for geographic data. When you choose to open a tabular file, the plugin tries to "guess" the two following things needed to convert the data to OSM:

To answer these questions, the plugin applies a set a of pre-defined patterns on each column name, assuming the first non-empty line is defining the file header. These patterns are currently defined for WGS84 and several "Lambert" West-European projections. When the plugin fails to detect any coordinate, you will have the error message "No valid coordinates have been found". If your file does contain geographic coordinates, please report it on the talk page.

Sometimes, the plugin will find coordinates, but will fail to guess the projection used. In that case, you will be offered to choose a projection among those known by JOSM:

Josm opendata projection popup.png

The easiest way is using a CSV file with text in "quotation marks", a semicolon as separater the the header "latitude";"longitude";"name".

KML/KMZ files

KML files generally express latitude/longitude in WGS84, so do not present the same problems as tabular files. However, the KML support is an extremely simple one, and may not cover all the existing files. If your file does not open, please create a new ticket on the JOSM bugtracker (component plugin opendata). Do not forget to attach or link to your file if you want the feature to be implemented.

GIS files (ESRI shapefiles, MapInfo files)

These files come from professional GIS. They usually embed the projection used in a separate text file, allowing the plugin to know it. However, some parameters known as "Bursa/Wolf parameters" are sometimes needed to convert coordinates to WGS84, and may be missing. In that case, please report it on the talk page, it should be possible to implement these parameters in the next plugin version.

The plugin allows you to bypass this limitation by using a "lenient" transformation without these parameters. A popup will allow you to choose this method or not:

Josm opendata projection error popup.png

The warning is here for a reason: this method can sometimes generate very large location errors (up to 1 km !).

Archive files

The plugin is able to search any of the other supported file formats inside Zip and 7z archives. When a single file is present, it will be opened transparently. When several files are found, you will have to choose which one to open:

Josm opendata zip.png

Download relevant OSM data

Diff

This feature is not implemented yet.

Modules

Modules are to this plugin what plugins are to JOSM: optional, independent JAR files enhancing the default behaviour, potentially written and maintained by different developers. The purpose of a module is to provide everything that is specific to a particular open data portal or geographic area. Indeed, given an open data portal, the corresponding module may, for each supported geographic data set:

  • provide an entry in "Open Data" menu allowing to download data from portal.
  • perform field mapping to convert raw data into a JOSM data layer.
  • provide an Overpass API request allowing to download relevant OSM data from OpenStreetMap database.

List of available modules

Name Country Town/Region Portal OSM wiki Supported data sets Comments Maintainer
be.bruxelles Belgium Brussels [1] 1 For test purpose
be.datagovbe Belgium Whole country [2] 1 For test purpose
fr.cg41 France Loir-et-Cher [3] 2 For test purpose
fr.datagouvfr France Whole country [4] WikiProject France/data.gouv.fr 9 Don-vip
fr.paris France Paris [5] 1 For test purpose
fr.sncf France SNCF [6] 1 Don-vip
fr.toulouse France Toulouse [7] Toulouse/GrandToulouseData 36 Don-vip

Installing a module

The installation process is similar to JOSM plugins. Open JOSM and select from menu Edit > Preferences, then there Open Data > Download list. Check the module(s) in the list. Now restart JOSM.

You can also copy manually the module JAR in "%APPDATA%\JOSM\plugins\opendata\modules" (Windows) or "~/.josm/plugins/opendata/modules" (Unix).

Enabling/disabling a module

Just like JOSM plugins, check or uncheck the module in Open Data plugin preferences.

Develop a module

This section explains how to develop your own module for you favourite portal. First of all, check the license is compatible with OSM ! If not, stop right here.

Setup directory tree

Writing data sets handlers

Writing main module class

Compiling/publishing the module