Mapsplit
Mapsplit is a command line Java application for splitting OSM data in pbf format into single tiles. The tool is written with a custom hashmap and bitmanipulation for an efficient and fast way to split the data in main memory. The code is open source and under public domain.
Contents |
Current status
Mapsplit is activly used to split OSM data for 3D rendering using OSM2World. Therefore bayern.pbf (225MB) is split in 2-3 minutes on a average PC with all data kept easily in main memory.
Downloading
The source code and a prebuilt jar can be obtained from github.
Compiling
Initiate an
ant
in the directory in which you'll find "build.xml".
Usage
Mapsplit has a simple command line interface. The quick usage description is given with the help option
mapsplit -h
The most basic way to run mapslipt will take an input.pbf file and write any tile that has changed in the last two days to output_dir
mapsplit infile.pbf output_dir
Options
There are also a number of options to influence the behaviour of mapsplit:
-
-v,--verbosewill print additional information during the splitting process -
-t,--timingwill output timing information
-
-m,--metadatawill include extra metadata into the written tilefiles. You usually won't need these extra data, however some tools (e.g. JOSM) depend on them for further processing
-
-b,--border=valenlarge tiles by val ([0-1[) of the tile's size to get a border around the tile.
-
-d=file,--date=filewill use the Data (written in UTF-8 by Java's DateFormater) in file as the base time. Every tile that has changed after this time will be written by the split process.
-
-s=n,w,r,--size=n,w,rtells the code how big the hashmaps size should be chosen. The defaults are quite large, however, if they don't fit for your input file you have to enlarge the hashmaps. With the verbose-flag above you can see the load of the hashmaps for your input.
Development
Mapsplit is reliable, however, there are still some missing features and there are still possible improvements that will be incorporated in the future.
A list of things that will probably be added in the near future can be found at github:
Notes
As the data gets written as it's read, mapsplit needs to keep many open file descriptors. In a default linux installation a user process may typically only open 1024 files at a time. For larger input data with many changes this limit will be reached quite soon. To extend this limit edit /etc/security/limits.conf and add the following line
heinz - nofile 4096
This will allow a process run by user heinz to open 4096 at once.
In case you run of of memory use
-Xmx2G
or appropriate.
