User:Galinette/FastOSM

FastOSM is an exploratory project for providing fast map download and upload when editing the map.

You can try it by downloading the editor here:

• Source code to be compiled with QT 4.3(using qmake)
• Windows executable

Don't forget to create an account here if you want to submit changes to the map. You need to set your login and password in the editor preferences menu.

Feel free to edit the map and consider it as a sandbox!

Introduction

FastOSM is an exploratory project aimed at providing better client/server performance. The main objective is to test various alternatives for improving the responsiveness of map download/upload when editing the map.

Currently, the experimented ideas are:

• Keep MySQL but use its geospatial extensions
• Tweak server performance (Index cache preload, storage on flash disk, ...)
• On the server side, do not wait having all the data ready to send before sending it on the network
• On the client side, do not wait having received all the data to parse it.

The client

Description

A client application has been developped to test the responsiveness of the server. You can browse the map, display internet tiles in the background (google maps, yahoo, osm), and edit the map. The map is loaded automatically in the view region. Your map changes are submitted by using the "Synchronize" menu action. Feel free to edit the map and consider it as a sandbox!

Try the client

You can download and try the client application here:

• Source code to be compiled with QT 4.3(using qmake)
• Windows executable

Create an account

To submit your map changes, you need to create an account first, and set it in the client(in the network tab of the preferences dialog)

Create an account here

The server

Server components

• Storage : MySQL, using geospatial extensions
• Data extraction and formatting : php5 script (not the best for performances, maybe ruby or python are better)
• Internet access : apache

The database scheme

All is based on a single table containing one row per object.

The SQL scheme:

CREATE TABLE IF NOT EXISTS `objects` (
  `id` bigint(20) unsigned NOT NULL auto_increment,
  `minzoom` tinyint(3) unsigned NOT NULL default '14',
  `rev` int(10) unsigned NOT NULL,
  `geometry` linestring NOT NULL,
  `user_id` bigint(20) unsigned NOT NULL,
  `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  `tags` text,
  PRIMARY KEY  (`id`),
  SPATIAL KEY `geometry` (`geometry`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8 AUTO_INCREMENT=15704504 ;

Each row is an object, corresponding to one of the following entities:

• Line (like non area ways in actual OSM data : highways, paths, ...)
• Area (like area ways in actual OSM data : forests, lakes, ...)
• Point (OSM nodes which have object information, such as name, traffic light, amenity, ...)

Table fields description

Here is a description of each field of the database:

• id : Identification number of the object
• minzoom : Minimal zoom level at which the object is to be displayed. Experimental, not used in the current client
• rev : Revision number of the object. An object is created with a rev value of zero, and rev is incremented at each object modification. Older revision number objects may be kept in the table or another history-dedicated table to keep track of map changes
• geometry : Actual object geospatial data, stored in MySQL binary format for spatial data. A spatial index is defined on this field, to allow fast extraction of all objects intersecting with a given rectangle. This is the key feature of geospatial extensions.
• user_id : Identification number of the user who has given the "rev" value to this object.
• timestamp : date of object creation or alteration
• tags : object OSM tags, stored in XML format (several name="value" pairs separated by semicolons)

Server to client data format

The data is sent in a simple XML format. All coordinates are given in 32-bit unsigned integers in square clamped mercator projection (equivalent to the OSM tiles coordinate system with zoom level equal to 32) The data is sent gzipped by apache is the client requests it in the http request header.

The XML format is quite easy to understand:

<mapdata version="2.2" maxlevel="32" u1="2175533056" u2="2175795200"
          v1="2816999424" v2="2817261568" 
          numobjects="237" querytime="0.0157289505005">

  <line id="895100" rev="0" mz="11">
    <tags name="Avenue Victor Hugo" highway="secondary"/>
    <pos rank="0" u="2174596294" v="2817514146"/>
    <pos rank="1" u="2174602438" v="2817518139"/>
    <pos rank="2" u="2174636640" v="2817540258"/>
    <pos rank="3" u="2174657939" v="2817554081"/>
  </line>

  <point id="107344" rev="0" mz="11">
    <tags name="Gare d'Austerlitz" railway="station"/>
    <pos rank="0" u="2175703407" v="2817102772"/>
  </point>

  <area id="1203062" rev="0" mz="11">
    <tags name="Square Tino Rossi" leisure="park"/>
    <pos rank="0" u="2175626381" v="2817237224"/>
    <pos rank="1" u="2175661669" v="2817203571"/>
    <pos rank="2" u="2175674403" v="2817187108"/>
    <pos rank="3" u="2175673732" v="2817185072"/>
    <pos rank="4" u="2175614196" v="2817240705"/>
    <pos rank="5" u="2175626381" v="2817237224"/>
  </area>

</mapdata>