€15.000 Funding for Software Development - Proposal/Bimonthly Report for June 2008
Bimonthly report OpenStreetMap for NLNet Foundation
Martijn van Exel, June 17th, 2008
The current OpenStreetMap API is not designed to handle history, changesets and rollback functions. Implementing this is an important step towards a larger user base and a lower barrier to entry for contributing to OpenStreetMap; the main goals of software development as defined in the development plan. Because an API change of this magnitude has great repercussions on a large part of the OSM software stack, it needs to be well prepared and discussed. This is why we decided to allocate a modest part of the NLNet funding to a 'hack-a-thon'.
The hack-a-thon brought together ten of the key developers in the OSM community from all over Europe. It was held in a London office provided by OSM founder Steve Coast on May 3rd and 4th, 2008. The NLNet funding was allocated to travel and accomodation expenses for the participants, who came over from Germany, Austria, the Netherlands and the UK. For an overview of expenses paid, see below.
What follows below is a write-up of the results of the hack-a-thon weekend, by developer Frederik Ramm. It was written right after the hack-a-thon. Discussion continued on the mailing list as a direct result of the hack-a-thon, and implementation efforts have been made towards the new '0.6' version of the API (current is 0.5).
- We won't quite do Postgres (etc) yet, but we will say goodbye to MySQL's MyISAM tables at least and switch over to InnoDB to achieve transactions. This is a big step forward as it will hopefully relieve us of inconsistencies.
- In the same spirit, version numbers will be added to the "current" table and returned by the API (and included in planet files etc.), and whenver someone uploads data, they must specify the version number that their update is based on. The update is then rejected if the server has a different version. Together with transactions, this will make sure that nobody inadvertently overwrites somebody else's edit.
- It will be possbile to upload a large set of database modifications in one go, and have this applied transactionally (i.e. either every- thing works or everything fails). This will greatly speed up edit uploads and also allow for secure https connections (where the connection set-up is time consuming and thus we could not support them with the current scheme that requires one connection for each object manipulation).
- We will force users to group changes into "changesets", and encourage them to specify something like a "commit comment" for a changeset. Changesets will have the affected bounding box associated with them in the database, and we will provide access methods to search and list changesets.
- We have not yet made plans for reverting changes; I think we have a consensus that reverting a change should be a "normal" operation that in itself constitutes a change (instead of somehow annulling a previous change in a matter-antimatter type of operation). With the ability to upload a group of changes, we do not have the pressing need to support reverts through an API call ("please revert changeset X") but instead can hope to outsource this so clever clients (that still have to be written). But a later API support is not out of the question.All this has led us to decide that we must step up to API 0.6 with aslight change in the existing calls. (Whether we should support somesort of backwards compatibility is still being discussed on dev; we haven't spent much thought on that.)
See also 
The project proposal is built on three development pillars. What follows is an overview of the status and a crude planning for each one.
Monitoring and Rollback
The hack-a-thon provided an excellent kick-off. Further refinements to the API definitions are now being discussed on the developers mailing list. A sandbox / prototyping environment for the new API version was set up and development is ongoing.
Planning: Another hack-a-thon focused more on implementation issues, to be held this summer.
Status: Starting up
Discussion is ongoing on the topic of platform choice. A former Google Summer Of Code project[] is currently being scrutinized for suitability / adaptability. This is geared towards ultramobile PCs like the Asus eee, rather than mobile phones / smartphones and J2ME, as initially envisaged.
Planning: Decide on a platform (J2ME or other?) in Summer. (re)define project planning shortly after that. Agree on requirements, developers, possibly during a hack-a-thon, this summer. Development in autumn / winter 2008
Various independent developments have popped up over the last few months. To name a few:
There has also been discussion recently on an annotation API (http://lists.openstreetmap.org/pipermail/dev/2008-April/009908.html)
Planning: The next step will be to consolidate these efforts into one production-ready application with an appealing front-end. This will be discussed with those involved on or before the State Of The Map conference to be held in Limerick in July. After that, resources should be allocated to integration, implementation and (re)design. This is as yet uncertain.
Funding allocated to OSM €15.000,00 Hack-a-thon Travel and Accomodation MvO € 474.10 Travel and Accomodation RD € 242.00 Travel and Accomodation GE € 380.00 Travel and Accomodation FR € 493.68 --------- TOTAL HACK-A-THON € 1589.78 Preliminary London visit Travel and Accomodation MvE € 346,89 --------- TOTAL PRELIM VISIT € 346.89 --------- TOTAL SPENT (as of June 17th) € 1936.67