Perl Scripts

From OpenStreetMap Wiki
Jump to navigation Jump to search

information sign

Software described on this page or in this section is unlikely to be compatible with API v0.6 (current version) deployed in 2009.
If you have fixed the software, or concluded that this notice does not apply, remove it.

Please make this notice more granular, so that only those apps that are actually broken with the 0.6 transition, are marked as such.

This page tries to give an overview of the Perl scripts that can be found on SVN. The idea is to have a short description here of what every script does.

If you are writing a new script, look at the [library], it contains functions to simplify many tasks. It also removes most of the distinction between different server versions.

/applications/utils/planet.osm/perl

This script takes an osm file like planet.osm and runs some simple statistics. A user name has to be given and then file and user statistics about nodes, ways and relations are given based on the field "user".
MOSTLY WORKS. This program counts the numbers of nodes, ways and relations per user in a .osm file. No history data is used. Just the field user in xml. Output is first general stats, then lists of users with numbers of nodes, ways and relations, unsorted. Then TOP lists are generated and displayed.
First line of program needs to read #!/usr/bin/perl -CIO. Syntax as stated within program doesn't work. Program is called via
./statistics2.pl input.osm
Calculates differences between two .osm files and generates a summary in HTML format as well as a graphic in PNG format. Requires GD.pm. Supersedes osmdiff.pl (version 1).
Another statistics program. Output is in structured format.
REQUIRES SLIGHT MODIFICATION. This program calculates the center of a user's activity in a given osm file/area for all users. It uses the field user and builds the means of longitude and latitude for each user. This information is added to the original OSM file and can be fed to a renderer, i.e. KOSMOS. The file is written to STDOUT. Users are labeled light, medium and heavy according to contributed nodes. Limits can be edited in source file. Example output of KOSMOS see here: KOSMOS CenterUserActivity.png
Does not correctly get negative values for lat/lon
LIMITED USEFULNESS. This program checks an OSM file for ways with open/unconnected ends or it checks certain ways for intersections on the same layer where no common node is present. The checks can be customized in an XML file. Output is HTML with positions, ways, other information and of course links to OSM and Openstreetbugs so you can inspect and correct data.
Program appears to function, but XML format is not documented. Supplanted by Osmose. Supersedes waycheck.pl (version 1).
LIMITED USEFULNESS. This program checks an OSM file for sparse areas that might need further mapping. Outputs two files, one in HTML with statistics, one in CSV format with slippy map information.
Not tested with extensive data. Program appears to function. Syntax is ./unmappedplaces61.pl input.osm output.htm slippy.csv.
NOT USABLE. Generates an HTML page indicating errors in a .osm file. Requires odd modules.
NOT USABLE. Another statistics program. Requires odd modules.
NOT USABLE. This script will import a .osm file into a database (mysql or postgresql). It can optionally only import the things within a bounding box, just exclude things within a bounding box. It can read from STDIN, or from a file, or download the latest planet.osm for you. If requested, can output the schema to use, or clean the database from a previous import.
The postgres support has been updated for the 0.5 API, but the mysql part may need some tweaks for 0.5
Sample use might be:
./planetosm-to-db.pl --dbtype pgsql --schema | psql -U osm -d osm
./planetosm-to-db.pl --dbtype pgsql Planet.osm
NOT USABLE. Intended to reorder nodes within ways.

/applications/utils/osm-extract

This takes a bounding box, and only outputs the parts of the planet.osm that are within it (much like the -bbox parameter to planetosm-to-db.pl). So, you can produce a mini planet.osm for just one area, without going via a database. Updated for 0.5
Oh, and it reads from STDIN if you want it to, so you can do
bzcat planet-061023.osm.bz2 | planetosm-excerpt-area.pl -bbox 50,-2,60,1 | bzip > mini-planet.osm.bz2
This script will probably work with API 0.6, as there doesn't immediately appear to be anything in it specific to the API.
UNLIKELY TO WORK. This script finds ways with certain tags on them, their segments and nodes, and nodes with certain tags on them. So, you could do an area excerpt for just the UK, then filter it through this to get all the railways and cities, then pass that straight to osmarender.
It doesn't support STDIN, as it has to make multiple passes over the file, so you should probably do an area excerpt first. Updated for 0.5
Has not been updated for API version 0.6.

/applications/utils/import

exclamation mark

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

osmtrackfilter/osmtrackfilter.pl

USAGE NOT RECOMMENDED. With this program you can read your tracks and filter them with different filters.
Even if it works, which it probably doesn't, it has many "TODO" items internally, most of them related to OSM.
common usage:
   osmtrackfilter.pl [-d] [-v] [--out-osm] [--limit-area]

<File1.gps> [<File2.sav>,<File2.ns1>,...] Other options which might be useful:

  • --man Complete documentation
  • --out-raw-gpx: write a raw .gpx file after conversion from .nmea, .gps, ...
  • --out-upload-gpx : write a reduced .gpx file
  • --out-osm: write a .osm file for loading into JOSM
  • --split-tracks: split Tracks if distance too long, speed too high, ...
  • --filter-against-osm : throw out all tracksegments found already in osm
  • --filter-dup-seg: throw out second time you drove a street
  • --filter-reduce: reduce number of points. (for example: reduce long straight lines)
  • --limit-area: use filter files to reduce area
  • --filter-clew: clews are produced if you have your GPS standing still
  • --filter-all: use filter files to reduce area
  • --draw_filter_areas: draw the (--limit-area) areas into GPX file
  • --generate_ways: try to guess way info(not working yet)
  • --fake-gpx-date: write faked date into GPX
  • --write-gpx-wpt: also include waypoints into GPX-file
  • --stdin: read from stdin
  • --stdout: write to stdout

/csv2osm

The old perl script has been moved to a 'perl' subdirectory, and a new 'ruby' version has been added.

The old perl version is not a straightforward file converter, rather it performs an immediate upload of the data to the OpenStreetMap server. This no longer works. It is only compatible with OSM API version 0.3 (very old) In any case, immediate upload of the data to the OpenStreetMap server is kind of frowned upon these days. Better to convert to a .osm file to review prior to upload using JOSM or a bulk_upload.py In fact this script will perform downloads to check to see if it's likely that the object already exists in OSM. If it thinks it might, it'll report the likely match, and defer the upload.

The ruby version of the script is a straightforward file converter. It does not perform any checks on the data, and does not perform an upload, but it does output data which is ready to be uploaded. It is up to the user to decide whether this is appropriate, and this always be carried out in accordance with Import/Guidelines

/applications/utils/perl_lib

This is a directory to include for all sorts of perl modules which might be useful for other osm-related Perl applications.

/Utils

  • Debug.pm
  • File.pm
  • LWP/

Stuff for downloading/mirroring planet.osm Files

  • Math.pm

/Geo

  • Geo/OSM/SegmentList.pm
  • Geo/OSM/Write.pm
  • Geo/OSM/MapFeatures.pm
  • Geo/OSM/OsmXML.pm
  • Geo/OSM/Planet.pm
  • Geo/OSM/Upload.pm
  • Geo/OSM/Tracks2OSM.pm

The [library] also lives here.

/Geo/GPX

  • Geo/GPX/File.pm

Read/write GPX Files


  • Geo/Geometry.pm

/Geo/Tracks

  • Geo/Tracks/Tools.pm

Handle Track data-structure in memory. Copy one track to another, split tracks, ...

Stuff to read different Track formats.

  • Geo/Tracks/Kismet.pm
  • Geo/Tracks/GpsBabel.pm
  • Geo/Tracks/NMEA.pm

/Geo/Filter

Stuff for filtering Tracks and OSM Data.

  • Geo/Filter/Area.pm

Filter data for a given Area. For now the area can be specified as a bounding box or country/region name.