Osmfilter

From OpenStreetMap Wiki
Jump to navigation Jump to search

osmfilter is a command line tool used to filter OpenStreetMap data files for specific tags. You can define different kinds of filters to get OSM objects (i.e. nodes, ways, relations), including their dependent objects, e.g. nodes of ways, ways of relations, relations of other relations.

Supported input and output formats are .osm format and .o5m format. To allow fast data processing, it is recommended to use .o5m format at least for input. The program osmconvert will help you converting other formats to .o5m. For example: ./osmconvert file.pbf -o=file.o5m

Download

These Downloads are available:

As usual: There is no warranty, to the extent permitted by law.

Program Description

To get a detailed description, please use the built-in help function of the program:

./osmfilter --help

This call will display a brief parameter overview:

./osmfilter -h

The following chapters demonstrate the most important use cases.

Object Filter

Filters of this category will always affect whole object (nodes, ways or relations). That means, these filter decide for every object if will be kept in the file or not.

Keep specific Objects and their Dependents

./osmfilter norway.osm --keep="highway=primary =secondary waterway=river" >streets.osm
./osmfilter switzerland.o5m --keep="highway=cycleway and lit=yes" >litcycles.osm
./osmfilter europe.o5m --keep= --keep-relations="route=bus" --out-o5m >bus_lines.o5m
./osmfilter bayern.o5m --keep="admin_level=6 and name=Nürnberger\ Land" -o=nbg_boundaries.osm

Each space character within a value must be preceded by a backslash.

Keep specific Object Type

To keep just a single object type specify a filter that only matches this, the other types need either be dropped or not matched by specifying an empty filter. To get only nodes with fixme=* use one of the following equivalent forms:

--keep-nodes="fixme=" --keep-ways= --keep-relations=
--keep= --keep-nodes="fixme="

Drop specific Objects

./osmfilter europe.o5m --keep="highway=" --drop="access=no" -o=good_ways.osm

Drop all Nodes, Ways or Relations

You can delete all objects of a specific type by applying one of these options (combinations are allowed):

--drop-nodes
--drop-ways
--drop-relations

Advanced Object Filtering

Ignoring interobject Dependencies

Usually all objects which are used by an object which is included will be kept in the data as well. For example, the filter --keep="highway=" will include not only all roads but also their nodes. The same applies to relations and their members. If you do not want these dependencies to be considered by the program, please specify this option:

--ignore-dependencies
Boolean Operators

To define complex filters you can use the Boolean operators and and or. The first has a higher priority than the latter, thus terms with "and" will be calculated first. You can supersede this order with brackets. Note that brackets must be separated by spaces from every other bracket, operand or operator. Example for a Boolean term:

--keep="place=city or ( place=town and population>=10000 )"
Comparisons

You may use six different types comparisons:

= != < > <= >=

The program will compare values ASCII-alphabetically by default. This may be not the right order for your language. The most noticeable difference is that all upper-case letters come before thee first lower-case letter.

For numeric values, the comparison will be calculated numerically. That means, "2" comes before "10" although "10" would be the first in alphabetical order.

Wildcards

When using the comparison operator "=", wildcards are allowed at the beginning and at the end of tags or values. For example:

--keep="name=main* and highway=*ary and source=*aerial*"

--keep="addr:*=*"
Special Tags

If it is necessary to compare meta data, the following expressions may be used as key values for the filter:

@id @uid @user

Tags Filter

Tags filter always refer just to certain tags, they never decide about the existence of whole objects. Use Tags filter to define tags you want to keep in the file or to be excluded.

Keep only specific Tags

./osmfilter a.o5m --keep-tags="all amenity=restaurant =fast_food" -o=food.osm
./osmfilter a.o5m --keep-tags="all highway= waterway= name=" -o=ways.o5m

Use this option with care. It can eventually filter out essential tags. For instance not keeping type= tag would break feather relation processing.

Drop specific Tags

./osmfilter a.o5m --drop-tags="oneway= name=" -o=plain_ways.o5m

Drop Author Information

For most applications the author tags are not needed. If you decide to exclude user name, user id, changeset and timestamp information, add the command line argument '--drop-author'. For example:

./osmfilter a.o5m --keep="route=" --drop-author -o=b.o5m

Drop Author Information and Version Number

If you want to drop not only the author information but also version numbers, use this option. For example:

./osmfilter a.o5m --keep="route=" --drop-version -o=b.o5m

Be aware of the fact that most programs will deny working with your data after you have deleted the version numbers.

Tags Modification

Sometimes it is useful to change certain tags to make further data processing easier.

Modify Values

You can specify values to be modified. For example:

./osmfilter a.o5m --modify-tags="highway=primary to =tertiary highway=secondary to =tertiary" -o=all_streets_are_small.o5m

This will make all primary and secondary roads to tertiary ones.

Modify Keys

Keys can also be modified:

./osmfilter a.o5m --modify-node-tags="amenity=fire_hydrant to emergency=fire_hydrant" -o=new_hydrant_syntax.o5m

Add new Tags

Same as with filtering, tag modification allows comparisons. Thus you can add redundant tags if this helps to simplify subsequent processing of your data:

./osmfilter a.o5m --modify-tags="maxspeed<=20 add speed_category=slow" -o=speed_categories.o5m

There is no check if there are already tags with the same key name. If necessary, use tags filter function to prevent possible collisions.

Getting Tag Statistics

To decide which items to keep and which items to drop, an overview over all tags used in a file would be helpful. There are four different output formats. The following sections will introduce these formats with the help of examples. All outputs have been clipped to 10 lines length.

List of all Keys

./osmfilter bremen.o5m --out-key
          5	Boden
          1	EE
         84	FIXME
          1	Fone
          7	StrVz
        113	TMC:cid_58:tabcd_1:Class
         78	TMC:cid_58:tabcd_1:Direction
        107	TMC:cid_58:tabcd_1:LCLversion
        110	TMC:cid_58:tabcd_1:LocationCode
         47	TMC:cid_58:tabcd_1:NextLocationCode

The first column shows the number of occurrences.
To specify a group of neighboring keys, you may use a wildcard at the end, for example:

--out-key=addr:*

List of all Keys, sorted by Occurrence

./osmfilter bremen.o5m --out-count
      28199	highway
      22794	building
      21365	name
      18754	created_by
      17751	source
      11842	addr:housenumber
      11835	addr:street
       5925	addr:postcode
       5724	foot
       5543	addr:city

List of a Key's Values

./osmfilter bremen.o5m --out-key=railway
        156	abandoned
          6	abandoned_tram
         16	buffer_stop
          2	bumper_stop
          9	construction
         31	crossing
         31	disused
          8	halt
        269	level_crossing
         38	platform

List of a Key's Values, sorted by Occurrence

./osmfilter bremen.o5m --out-count=railway |head
       1570	rail
        269	level_crossing
        192	tram
        182	tram_stop
        156	abandoned
        126	spur
         81	razed
         38	platform
         31	crossing
         31	disused

Parameter File

The command line argument can get very long if you specify complex filters. Please use a parameter file instead and refer to this file with --parameter-file=. For example:

./osmfilter europe.o5m --parameter-file=my_parameters >line_1.o5m

File "my_parameters":

-v

--keep=

--keep-relations=
all
route=bus
line=1

--drop-tags=
operator=
direction=

--out-o5m

Empty lines are used to separate the parameters. Linefeeds within parameters will be converted to spaces.

Benchmarks

Planet Pistes

16GB ram, 7200rpm disk

 $ time ./osmfilter new-planet-latest.o5m --keep="piste:type= or aerialway=" > pistes.osm

 real	12m5.413s
 user	4m30.465s
 sys	0m28.742s

Planet Waterways & Naturals

 $ time ./osmfilter planet-121207.o5m --keep= --keep-ways="natural= or waterway=" --keep-relations="natural= or waterway=" -o=planet-natural-temp.o5m 

 real	8m6.760s
 user	5m27.622s 
 sys	0m35.670s

16GB, SSD