Osmosis
| Sub pages |
|---|
| Installation |
| Quick Install (Windows) |
| Detailed Usage |
| Examples |
| Benchmarking / Tuning |
| Development |
| Writing Plugins |
Osmosis is a command line Java application for processing OSM data. The tool consists of a series of pluggable components that can be chained together to perform a larger operation. For example, it has components for reading from database and from file, components for writing to database and to file, components for deriving and applying change sets to data sources, components for sorting data, etc. It has been written so that it is easy to add new features without re-writing common tasks such as file or database handling.
Some examples of the things it can currently do are:
- Generate planet dumps from a database
- Load planet dumps into a database
- Produce change sets using database history tables
- Apply change sets to a local database
- Compare two planet dump files and produce a change set
- Re-sort the data contained in planet dump files
- Extract data inside a bounding box or polygon
Osmosis can also be included as a library in other Java applications.[1]
Contents |
Current status
Osmosis is in a state where the majority of the codebase is unlikely to change drastically. New functionality will be introduced over time in the form of new tasks and plugins but existing functionality is unlikely to be removed. All bug reports welcome.
The current released version only supports the API 0.6 OSM data model.
Downloading
Latest Stable Version
The stable releases below should be used whenever possible.
- http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-latest.zip
- http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-latest.tgz
Development Build
These should only be used if you require access to the latest features or bug fixes. No guarantees are made regarding the stability of these releases.
Regular builds are created on the continuous integration server. These include the git version number to allow builds to be easily identified.
Java 1.5
For those that can't install a Java 1.6 runtime, a 1.5 compatible release is available below. This is known to be an issue with some older Mac systems.
- API 0.5: http://gweb.bretth.com/osmosis-0.24.1-java5.zip
- API 0.6: http://osmosis.achmyr.com/api0.6-java1.5
API 0.5 Support
The 0.35 release was the last release to support API 0.5.
- http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-bin-0.35.zip
- http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-bin-0.35.tgz
- http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-0.35.zip
- http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-0.35.tgz
DB Schema
(see Osmosis/PostGIS_Setup)
Database schema creation scripts matching the production API are bundled with the Osmosis distribution within the script/contrib directory.
Database schema creation scripts for the Osmosis PostGIS "simple" schema and "snapshot" schema (osmosis 0.39 ff) are bundled with the Osmosis distribution within the script directory.
Source Code
The Git repository is hosted on GitHub at: https://github.com/openstreetmap/osmosis
(More details can be found on the Osmosis/Installation instructions sub-page.)
Usage
Detailed Usage
Full details of all tasks and their usage are available at the following links:
- Latest Release Detailed Usage (recommended).
- Development Detailed Usage.
Usage details for older versions are available below:
- 0.35 Detailed Usage (this is the last version to support reading OSM v0.5 files).
- 0.39 Detailed Usage.
- 0.40 Detailed Usage.
- 0.41 Detailed Usage.
- Development Detailed Usage.
To maximize performance, refer to Tuning and Benchmarking.
There is also a frontend for the use of Osmosis named OSMembrane.
Example Usage
(More on the sub-page Osmosis/Examples.)
Import a planet file into a local PostgreSQL rails port database.
osmosis --read-xml file="planet.osm" --write-apidb host="x" database="x" user="x" password="x"
Export a planet file from a local PostgreSQL rails port database.
osmosis --read-apidb host="x" database="x" user="x" password="x" --write-xml file="planet.osm"
Derive a change set between two planet files.
osmosis --read-xml file="planet1.osm" --read-xml file="planet2.osm" --derive-change --write-xml-change file="planetdiff-1-2.osc"
Derive a change set between a planet file and a local PostgreSQL rails port database.
osmosis --read-xml file="planet1.osm" --read-apidb host="x" database="x" user="x" password="x" --derive-change --write-xml-change file="planetdiff-1-2.osc"
Apply a change set to a planet file.
osmosis --read-xml-change file="planetdiff-1-2.osc" --read-xml file="planet1.osm" --apply-change --write-xml file="planet2.osm" or osmosis --read-xml file="planet1.osm" --read-xml-change file="planetdiff-1-2.osc" --apply-change --write-xml file="planet2.osm" for the older version
Sort the contents of a planet file.
osmosis --read-xml file="data.osm" --sort type="TypeThenId" --write-xml file="data-sorted.osm"
The above examples make use of the default pipe connection feature, however a simple read and write planet file command line could be written in two ways. The first example uses default pipe connection, the second explicitly connects the two components using a pipe named "mypipe". The default pipe connection will always work so long as each task is specified in the correct order. Each task pushes its output stream(s) onto a stack, and each task that requires input stream(s) takes them from the stack. This stack-based approach is the reason why the above "--apply-change" example first reads the change file, then the data file - the --apply-change task will then first retrieve the data file from the stack and then the change file.
osmosis --read-xml file="planetin.osm" --write-xml file="planetout.osm"
osmosis --read-xml file="planetin.osm" outPipe.0="mypipe" --write-xml file="planetout.osm" inPipe.0="mypipe"
Extract an area based on a Osmosis/Polygon Filter File Format file:
osmosis --read-xml file="planet-latest.osm" --bounding-polygon file="country2pts.txt" --write-xml file="germany.osm"
Extract only those ways that have certain tags. Specifying --used-node only extracts those nodes that are used in the filtered ways. Relations stay untouched.
osmosis --read-xml city.osm --way-key-value keyValueList="railway.tram,railway.tram_stop" --used-node --write-xml city_tram.osm
On larger files (e.g. a planet file), this may result in a segfault due to memory limitations. In this case, passing idTrackerType=BitSet as a parameter to --used-node may help:
osmosis --read-xml city.osm --way-key-value keyValueList="railway.tram,railway.tram_stop" --used-node idTrackerType=BitSet --write-xml city_tram.osm
Note that --way-key-value and --node-key-value cannot be used in one run. Instead, create two (or more) files in several runs and merge them together afterwards (yes, for n files you need n-1 merge commands):
osmosis --rx 1.osm --rx 2.osm --rx 3.osm --merge --merge --wx merged.osm
Only 0.5 tasks are available from version 0.22 onwards.
Extracting bounding boxes
One of the basic use cases is extracting a subset of OSM XML from a large downloaded file. You might have downloaded the world OSM (planet.osm) and want to extract some region, or maybe you have a country extract downloaded but only want one city. The simplest way to do that is by extracting a rectangle.
An example that extracts Nürnberg and some area around it from Germany - or maybe world - OSM:
bzcat downloaded.osm.bz2 | osmosis --read-xml enableDateParsing=no file=- --bounding-box top=49.5138 left=10.9351 bottom=49.3866 right=11.201 --write-xml file=- | bzip2 > extracted.osm.bz2
This approach uses bzip2 directly for extracting and compressing, as it is claimed to be more effective than the built-in archiver.
The easiest way to get coordinates is to use the map on www.openstreetmap.org. Doubleclick the spot you want coordinates for, then click "Permalink" and grab the data from the URL field.
Windows notes
bzcat is the tool to use even on Windows if you want to have good performance. However, /dev/stdin is not a valid path on Windows. Use CONIN$ instead.
Getting Help and Reporting Bugs
The best place to get answers to questions on osmosis usage is the osmosis dev mailing list.
http://lists.openstreetmap.org/listinfo/osmosis-dev
Please discuss potential bugs on the dev mailing list before creating new tickets. If you're confident you have discovered a bug, it can be logged in the OSM trac system against the "osmosis" component:
http://trac.openstreetmap.org/query?component=osmosis
See also
- OSMembrane - a GUI for Osmosis
- OsmChange - file format used by osmosis to describe differences
- Osmosis/WritingPlugins - To write your own tasks, see Osmosis/WritingPlugins
- Osm2postgresql embeds osmosis and can install postgresql & postgis and have your OSM data ready with a single command
- Mapsplit, Splitter, osmconvert, osmchange, Osmcut, pbftoosm are some other tools to split a map into smaller tiles.
- Osm2pgsql also can do import to postgis db
Development
Details on osmosis development are available at Osmosis/Development. Visit there for documentation on the internal structure of osmosis, design decisions, steps to configuring development environments, etc.
Notes
The minimum supported Java version is 1.6. Osmosis makes use of some java.awt.geom classes that only appeared in 1.6, if these are re-written to use 1.5 functionality it may be possible to return to the previous minimum of 1.5. Osmosis utilises generics and java.util.concurrent features requiring Java 1.5 as a minimum.
The built-in bzip2 (.bz2) performance is much slower than the gzip (.gz) built-in Java one (Java gzip utilizes native code to improve performance). However, the trade off is that bzip2 creates much smaller compressed files. Also recent planet downloads use features of bzip2 which are not supported by the built-in bzip2 library (streams). For working with these, and for faster processing of any larger files, it is suggested to use your platform's native bzip2 implementations and read and write files from /dev/stdin and /dev/stdout respectively.
pbzip2 -d -c planet.osm.bz2 | bin/osmosis --rx /dev/stdin --wx /dev/stdout | gzip -c > planet-same.osm.gz
See also /Benchmarking
Under Win 7 64 bit it may be necessary to run the .bat file from a command prompt. The underlying osmosis.bat file may need to have the full path name for java.exe inserted.
