OSM Composer/Manual

Manual page for OSM Composer up to Version 0.60

Contents 1 Basic Usage 1.1 Regions 1.1.1 Maximum region size 1.2 Main Window 1.3 Settings 1.4 Generating maps 1.4.1 MapSource Integration 1.4.2 Garmin device upload 2 Customizing maps 2.1 Ignore list 2.2 TYP files 2.2.1 Creating a TYP file 2.2.2 No TYP file / External TYP file 2.3 mkgmap rules 2.3.1 Predefined rules 2.3.2 Importing rules 2.3.3 Editing rules 2.4 Replacements 2.5 Routes 2.6 Statistics 2.7 Levels - Problem with Contourlines below Polygons in Mapsource 3 General 3.1 List handling 3.2 Editing 3.3 Logging 4 Troubleshooting 5 How to start working if you already have a map-features.csv list and a typfile

Basic Usage

Regions

Each regions is a rectangular area. The coordinates are the same as used in the slippy map/Export feature. The easiest way to enter a region is copying a permalink URL into the bottom URL field and pressing the "Parsen" button.

• Regions can be included and removed from the generation process by checking the "Aktiv" status. There is no limit to the number of regions.
• Downloading Regions
  • The field "Kachelgroeße" indicates the size of the tiles for downloading in degrees. You can use larger tiles in sparsely mapped areas and smaller ones in densly populated/mapped areas. Composer sends several tile requests in parallel and smaller tile sizes tend to speed up downloading as the answering time of the server for each tile decreases. If you get server errors, try reducing the tile size. I use 0.05 for densely populated areas and 0.1 for backcountry.
  • "Updateinterval" is the number of days the data is considered good before it is re-downloaded from the OSM server. My primary mapping area for example is updated daily while I consider once a week sufficient for my living area.
• Reading part of planet files
  • Composer will call osmosis to cut the area within the coordinates from a larger planet or country file and then import it
• Reading prepared files
  • You can add a local data file that you have cut and prepared in some way. Composer will ignore the coordinates and process the whole contents of this file instead. The data is not cut in any way in this mode, so make sure the file does not contain too much data or you might experience an out of memory exception.

Maximum region size

Experiments with different region sizes have shown that Composer V0.51 with VM setting -Xmx1600M can handle a region of about 1°x1° and input files of up to 200MB OSM data safely. Anything above this is likely to produce out-of-memory problems. In my experiment the limiting factor was srtm2osm, which crashed generating a 2°x2° area. For a detailed hiking map, the contour lines are necessary, so I guess this is about as large as a region can get. The answer beyond that is defining multiple regions which will be combined into a single map anyway (only if both are active at the same time). Since v.0.52 srtm2osm gets called with parameter --large. Now the limiting factor should in theory be mkgmap. So about 550MB of contourlines in osm format, OR about 300MB of OSM data should be processable (whichever the greater) (this depends on your mkgmap rules, the more tags you include, the smaller). About 1024 MB must be set in java memory request setting, so that mkgmap doesn't crash on creating the map.

Main Window

The buttons above the region list offer shortcuts to the most important lists in Composer. The drop down box allows you to focus your work on one type of map object. When you press the buttons "Gerät" or "Umsetzungsregeln", the lists will not show all items but only those of the selected type. The options are

• POI: Work on nodes
• Flächen: Work on polygon areas
• Wege: Work on ways
• Wichtige POIs: Nodes marked as visible in Gerät and as priority in the rules

Settings

• Working directory: Define a working directory for all downloaded and generated files. If this is empty, the start directory is used.
• Icon path: Define a path where all icons are stored/read from. Composer must be restarted for a new path to take effect.

Generating maps

MapSource Integration

Composer can prepare map data so that it works with Garmin MapSource. This can be enabled in the Settings. The data files are generated by mkgamp and cgpsmapper.

To add the generated map into MapSource, Composer generates a .reg file with all required entries and installs them into the registry. You can look up the .reg file in the working directory.

MapSource must be closed while Composer works. If it is running you will get a warning message and generation will not start.

The overview map created by the current version of mkgmap is faulty at the moment. Composer renames and patches the files to make them work with MapSource, but they might look differently when created manually on the command line.

To uninstall the map, simply disable MapSource integration in the settings, upon the next generation run the entries will be removed from the registry.

Once the registry entries are in place, MapSource requires the map data to be present. If one of the files is missing, e.g. due to a problem in the generation process, MapSource may crash upon startup. This can be repaired by

• running a successful generation, thus recreating the data
• disabling MapSource integration and running the generator, thus uninstalling the map

Garmin device upload

Composer calls sendmap20 to generate the final map.

• If the checkbox "Upload" on the main window is checked, the map is uploaded directly onto your device.
• Otherwise the file gmapsupp.img is created in the working directory for manual installation.

Customizing maps

Ignore list

There is an ignore list of tags that are not useful for rendering and are removed by Composer for faster processing, smaller data size and more readable statistics. If you need to process one of these tags, contact me to take it out of the ignore list. Currently the following tags are removed:

"created_by", "source", "note", "opengeodb", "admin_level", "is_in", "addr", "todo", "population", "onkz", "url", "comment", "converted_by", "history"

TYP files

The list "Gerät/TYP" contains all objects displayable on the Garmin device and their IDs. It defines the available building blocks and is the basis for all rule editing and customizing. It is very helpful to enter a description of what the object means and/or how it is displayed here as this text will be shown to select and handle the object. Optionally you can define a custom layout for each item that will be compiled into a typ file to accompany the map. The objects to be used need to be marked as visible "Anzeigen". The unused state is useful to mark items that cannot be used/modified on the device (e.g. the predefined map background) or obsolete items during a rework.

You can use this in two ways.

Creating a TYP file

Composer contains a TYP file editor. The button "Gerät/TYP" opens a list of all map objects defined for the device. Every object needs to listed here. It is possible, but not required to customize the design. Composer generates a TYP definition file from all entries with custom design, compiles it with cgpsmapper and includes it with your map.

There are the following possible customizations:

• Name
  • Set the name shown on the device when the arrow is hovering over the object
• Line type
  • Works for ways (polyline) and filled ways (polygon)
  • set a line width and a border width
  • set a line colour and a border colour
• Image
  • Line pattern for ways (32 x n, two colours)
  • Fill pattern for filled ways (32x32, two colours)
  • Icons for nodes (point) (max 16x16, 128 colours)

See the documentation of cgpsmapper for details and limitations of the TYP definitions. Currently Composer supports only one colour set (day colours). If you are using a topo map at night odds are that you are in trouble and the colours of your GPS are the least of your worries. :-) (once 3byte Polyline/Polygon support is added to mkgmap, the limitation for ways and filled ways will be the same as the ones for POIs. Some garmin maps like Topo Germany V2 use this already. 3byte definitions will ONLY show up on map if used with typfile.

No TYP file / External TYP file

If you don't want to use a TYP file or you already have an external TYP file, you can use this list to indicate if and how the various Garmin objects are being rendered. This will help you in getting an overview of what your mkgmap rules are doing.

You can use the test map of mkgmap to check what your device can render and enter a description of the line or icon in "Ansicht". Check "Anzeigen" to indicate the feature is usable in mkgamp mapping rules.

mkgmap rules

Predefined rules

• If you want to create your map with mkgmaps default design, activate the checkbox "Standard Regeln" in "Einstellungen"
• If you want to use a ready made set of mapping rules, deactivate the checkbox and enter the path to your rule file at "Externe mkgmap Regeln" in "Einstellungen"

Please note that Composer just adds as a dumb compiler shell in this case. You will not be able to use the internal map conversions and the route support as they depend on adding additional mkgmap rules.

Importing rules

The application comes with some demonstration files I use to create a map for hiking/trail riding. You can start your own work from these or import some other .csv files, e.g. the default files included with mkgmap in the /resources directory. If your map-features.csv uses a custom TYP file, it is recommended that you import a matching file like garmin_feature_list.csv first with "Garmin Objekte importieren" so Composer knows what objects are defined for your device. This will replace the list in "Gerät/Typ". You can import your map-features.csv (or any other working mkgmap rule file) with "Kartenumsetzung importieren". This will replacethe list "Abbildung" and you can work on it with the editor.

Editing rules

If you want to customize the layout of the maps for your needs or for your device, I recommend that you use Composer's editor for managing the mapping rule files used by mkgmap. Composer will generate a new customMapping.csv file from the editor data and use it to compile the map.

The Button "Abbildung" opens a list of all defined tags for points, ways and area ways and the target device objects assigned for visualizing them. Design the map you want here. If an entry is shown in red after generating a map, this means that the assigned target device item is marked as unusable or missing and you need to look for an alternative.

Instead of assigning Garmin objects to rules in the drop down box, you can also select the Object in the list Gerät and select "Use in rule" from the context menu. If the rule dialog is open, the drop down box will be set to use the selected object. This is often faster than browsing the drop down list.

The order of the rules is important. Mkgmap will use the first matching rule and ignore all others. You can display the list in different sort orders, but it will always be generated in its fixed order as shown when your first open it. You can change the position of an entry in the list by moving it up and down with the cursor keys:

• Ctrl-Up: Move rule up, increase priority
• Ctrl-Down: Move rule down, decrease priority

Replacements

The "Ersetzung" button opens a list of replacements. OSMap Composer parses the data after downloading or reading it from an input file. The replacements are performed on the OSM data before saving it. This means that the data has to be downloaded/read again in order to see changes done to anything in "Ersetzung".

You can specify one or two tags to check. If you select two, both conditions must be met. The value of the tag is checked using Java regular expressions. If you just want a text comparison, simply enter your text. If you want to do a more complex comparison, you can use the full power of the regex syntax.

For example the condition

oneway=yes|true

would find both correctly and mistagged dead end roads.

If it finds one of the tags listed, it executes a replacement action:

• "Outline erzeugen" creates a copy of a way, strips all tags and assigns a temporary tag "outline" with the given value. This tag can be used in a custom mkgmap mapping rule. In this way you can draw a second polyline on top of the real way, giving you more possibilites than the rather limited set of graphics on Garmin devices. Possible uses:
  • Outlining areas in a different colour
  • Striking through blocked roads by overlaying them with a thinner object like railroads (without typ file) or a special way style for blocked/prohibited roads.
  • Mark ways by a combination of two existing ways to conserve IDs
• "Namen hinzufügen" adds an additional name tag or extends an existing name tag with the given text to show properties by text if they cannot be visualized by line style or colour
• "Tag setzen" adds and additional tag with custom content
• "Tag austauschen" removes the tag or tags that have been checked and replaces them by a new custom tag

If there are several matching rules for one way, all of them are executed in the order shown in the list. You can change the order by moving rules up and down with Ctrl-Up and Ctrl-Down. If you check "danach abbrechen" no further rules are executed on a way if the current rule did match the tags.

Limitations of available IDs

Notice however that as long as mkgmap doesn't support 3-byte typfile support there is quite a strict restriction on the number of different polylines that you can incorporate into your map. If you use "Outline erzeuge" you have to options: Either you overlay an existing way over it, like railroad tracks (this should still be easy to identify then, wheter the condition is met or if it's a genuine railroad, OR you can copy it to a new Polylines style. While the first option doesn't reduce your limited polylines (0x00 to 0x2b are acceptable values in hexadecimal for Mapsource, while 0x2c to 0x3b will ONLY show up on your unit (tested on Vista HCx) but not in Mapsource (valid for version 6.13.7). 0x17 doesn't show up for me at all, while 0x20 to 0x25 is reserved to contourlines/depth lines and also not freely configurable. For Polygons basicaly the same applies but the all Polygons up to 0x5f show up in Mapsource and 0x4b is the map background and therfore not freely configurable. For POI the possibilities are much larger as 3byte support is already implemented. POI classified wrongly will however show up in wrong categories when searching for features. Therefore including new things as POI into the map that aren't standard for garmin maps, i.e. wlan hotspots) a good category has to be assigned (fitting would for example other/communications). I don't know wheter/how it's possible to defines ones own categories for the search function.

Routes

Composer can process route relations (tagged as type=route) and mark them in a map. You can set a filter in settings what routes your are interested in. The default is hiking routes (route=foot). The setting is a regular expression, so you can specify multiple route types e.g. "foot|bicycle".

Composer will read the description of all matching rules from the OSM data and display them in the route list. Entries contained in the last generation run are shown in black, older entries are in gray and newly found entries are in green. As the OSM data contains no useful information on how to render a route, you need to configure this in the route list and set the "active" checkbox to make the route appear on the map.

There are several ways to mark a route:

• by name
  • The specified text will be added in brackets to the name tag.
  • This is only done for ways above a minimum length defined in settings. Default is 300m. Short ways are not named to avoid clutter.
• by highlighting
  • The route will be marked by drawing a copy of the way under it. This marking should be wider than the way itself, otherwise you won't be able to see it
  • the copies are tagged with a special tag "route_marker"
  • There is a general marker that is used for all routes. The tag value for this marker can be defined in settings. The default is "background" and there must be a matching in rule in the mkgbmap rules to make it visible. I use a white line 2 pixels wider than tracks and minor roads.
  • There is an individual marker for each route. It will be drawn above the general marker and below the way itself. Select a way rule in the drop down box to use in the route configuration dialogue. I use coloured squares to indicate which coloured markings belong to this route.
• by icon
  • Composer will select a node near the middle of the way and tag it with a "route_marker" tag.
  • This is only done for ways above a minimum length defined in settings. Default is 500m. Short ways are not tagged to avoid clutter.
  • If there are multiple markings on a way, they will use different nodes.
  • Select a point rule in the drop down box to use for rendering

After setting up/changing routes you will have to run the map update again, as the changes are done directly in the OSM data.

During generation, Composer evaluates how many meters of this route are in the processed data. It will also show its findings in the statistics.txt. Note that this number can be much smaller if your regions show only a subset of the total extent of the route.

Statistics

Composer runs some statistics on used and unused tags. The statistics are written in the file Statistics.txt after generation.

In additon to the global ignore list, the following tags are not counted in the statistics:

"name", "ref", "int_ref", "layer"

Levels - Problem with Contourlines below Polygons in Mapsource

See here until a definite solution is implemented: Talk:OSM_Composer#mkgmap_--levels

General

Data handling

• The data in the lists is saved to disk when you quit Composer or when you select "Zwischenspeichern"
  • Do this occasionally to avoid data loss in case there is a crash (unlikely but possible)
• If you make a major mistake damaging data, you have the option to end Composer with the menu "Datei/Abbruch ohne speichern" to keep the last saved set of data.
• The .tbl files of Composer are basically comma delimited lists and may be modified with an ASCII editor with care.

List handling

In every list in Composer, you have the following tools available:

Changing the list layout

• Column width: Change by dragging the column end
• Column order: Change by dragging around the columns
• Adding and removing columns: Right-click the column header and select option from menu
• Make layout changes permanent: Right-click the column header and select "Layout speichern".

Filtering

• Set filters to narrow down the number of items you are working on or find specific entries
• Right-click in the column you want to filter and select "Neuer Filter" from the menu
• Select "Letzten Filter zurücknehmen" from the right-click menu to undo the last filter, keeping previous ones

Change one data value for several entries simultaneously

• Select several entries in the list
• Right-click in the column you want to change
• Select "<Column name> ersetzen" and enter the new value
• I don't have to mention that you can destroy your data very carefully this way if you use this carelessy, do I?

Editing

In a list you have the following keyboard shortcuts

• Insert: Create new entry
• Shift-Insert: Copy current entry
• Return: Edit entry
• Delete

There is an entry in the right-click menu for each operation.

In an editing dialog

• Data fields for files/paths
  • You can drag a file into the text field, copying the path there
  • The button "..." will open a file selector
• Drop down selections with Button "..."
  • If nothing is selected in the drop down box, the button creates a new entry for the list and uses it.
  • If a value is selected, the button will open the entry for editing.

Logging

There is a log that notes down some details on what Composer is doing. You can call up the log with the menu "Datei/Log". If there is an error or exception detected, the log will popup by itself and show the problem in red.

Do not increase the log level above the default setting, the additional info is internal and will not help you in understanding what Composer is doing.

There is also a file "commands.log" produced in every generation run. It contains all external command line calls for debugging or re-using them manually.

Troubleshooting

Composer runs a complex generation process with many external tools and variables. Things may go wrong, especially until all surroundings are configured properly. Here's some hints on how to find out what's going on.

• For first installation: Go through the settings and check whether all paths to external tools are valid.
• Look into the internal log. You will find some additional information on what Composer is doing and what steps it thinks it has completed successfully.
• Look into "commands.log" and check whether the external calls look plausible. You can try to run them one at a time on the command line and see whether they work there and what console output they produce.
• Look at the data flow diagram on the main page. Check the working directory: which of the files have been created? Are some missing, way too short or have an old file date?
• srtm2osm and Composer require an internet connection. Does your firewall allow them through?

How to start working if you already have a map-features.csv list and a typfile

• Follow the basic usage guides.
• Instead of editing "Geraet/TYP" and/or "mkGmap Regeln" import your map-features.csv using "Daten" -> "Umsetzungsregeln importieren". Then let the map generate.
• After the map is generated, register it in Mapsource, and exchange your typfile with the one created just before. Now however your typfile will still miss everything from the changes you have made under "Ersetzung" and "Routen". Therefore you now have to adapt your typfile including the definitions that you remapped polylines/polygons/POIs with the "Kopie/Overlay erzeuge" rule. or "Icon Einblenden". I'm not sure how the Tag rules actually work. Maybe you have to change the tag manually in your typfile now (please correct if you understand it better).
• I'm too unsure whether the "Geraet/TYP" button now mattters at all, or if you can forget about it completely.
• Limitations of going this way: As there is no possibility to import .typ files yet, the rendering shown will not correlate with the rendering shown using your own typfile. I overcome this by afterwards analysing the typfile created by osm composer with the maptk freeware (windows and linux supported), opening the analysed typfiles (your one too) and copy/pasting the missing or better looking definitions into the typfile you want to use. General knowledge about how the .typ files work is needed for this process.
• For a cycling/mtb specific rendering you could download the following files (under progress). (Is it okay with with you nop if I upload a changed tbl set? in a package including a csv file and a typ file to SVN? This way people could just overwrite the default installation of osmcomposer and directly have a working edition for composing special interest maps. I will not include any jars, so people are forced to download the latest version).
• If you get any import errors please look here: http://wiki.openstreetmap.org/index.php/Talk:OSM_Composer#Kartenumsetzung_Importiergen_java.lang.ArrayIndexOutOfBoundsException:_5