User:Grahamjones

	This user submits data to OpenStreetMap under the name grahamjones.

I support OSM by being a
member of the
OpenStreetMap Foundation
Are you?

Mapping

I discovered OSM in April 2007 and my daughter Laura and I decided to map our home town of Hartlepool by bicycle. We have got the main roads done by a combination of bike and car, and need to fill in the gaps where the residential areas are. Laura has got bored with mapping now that she does not need it as an excuse to be allowed to cycle on roads. This means that my Son Benjamin and I have been doing a different bit of housing estate ourselves every nice Sunday morning - nearly there - see Hartlepool for a picture showing progress.

I have also added quite a few footpaths and bridleways in the area - for example the Durham Coastal Path from Hartlepool to Seaham, and some routes around the North Yorkshire Moors.

Speciality Maps

I have had an idea for some time to set up a web site to host speciality maps (e.g. historic places, power stations, supermarkets etc.). I already have a few of these set up as static maps, which were generated on my home server maps3.org.uk and maps3.org.uk/osm_opendata. The problems with these are:

• They only update when I decide to do it.
• They are only for the UK.
• There is no 'edit' facility - a speciality map is really good for spotting things that are missing form the map, but it is a pain to go and add them.
• They appear very slow because they are hidden behind my domestic broadband connection, which is nice and quick downloading, but very very slow uploading data.
• There is a steep learning curve for a less technical person to do their own map rendering, which discourages people from setting up their own special interest maps.

There have been a few discussions on the talk-gb mailing list recently about these sort of maps, and they seem popular (at least there are a couple of people interested in each one!). So I have decided to have a go at setting up something a bit better.

The goal of the project is to make it easy for less technical people to create and maintain their own specialised map overlays and display them on the web.

The following sections describe what I propose to set up, and how to do it.

Vision

The goal of the project is to provide a simple system to allow people to develop and deploy speciality map overlays of OSM data (e.g. an 'industrial history' overlay that can be drawn over another base map). It will include automatic updating as the underlying data changes, and provide specialised editing facilities targeted at the particular specialism of the map.

The benefits to end users and maintainers are:

End Users

I envisage this system being used in two ways - people particularly interested in the speciality will find it easier to see where the points of interest they are interested in are missing, or incorrectly tagged, allowing them to improve the map from their point of view. The other use is as a simple (minimum learning curve) introduction to OSM for new users, by providing an editor with limited functionality that is tailored towards the type of map being used.

Maintainers

I envisage this system being maintained by people that are half way between casual mappers and serious developers - that is, the tools should be easily set up by someone without a working knowledge of the underlying technologies, but who is willing to invest effort in setting up the relevant mapnik stylesheets to produce the maps. This will allow people with less technical knowledge to create their own custom maps.

Obviously the initial development will require people with the detailed knowledge of the technologies, but the tools will be packaged to make their use as straightforward as possible for the casual user.

Similar Projects?

I do not know of any projects that are the same as this, but would like to know in case I am wasting my time. There are specialist outdoor (freemap) and cycling maps, plus I have seen one that concentrates on public toilets, which is similar to this, but extremely specialised. A large list can be seen on the OSM Wiki.

Maposmatic and Townguide can do printed output, but not for custom maps.

Therefore I think there is a gap, unless someone can tell me otherwise?

Proposal

End User Look and Feel

• The web page presented to the user will contain a drop down list of the speciality maps provided on the server (It will have the capability of linking directly to a particular map).
• The page will have a number of tabs across the top to select particular content (Map, Map Key, Edit, About).
• It will provide a facility to produce printed output of the selected map.
• The page will link to a detailed description of how the system works, and how someone would go about adding another map to it.
• The page will include a 'news' section listing when maps have been updated, proposals for new maps etc.
• There will be an 'administration' page that map maintainers can log into to update their maps without needing access to the main server file space.

Map Creation Workflow

The way someone will create a new map is:

• Download a sample mapnik stylesheet from the server, and customise it to draw the things they are interested in.
• Upload the stylesheet to the server.
• Ask the server to create a sample map. If everything works, the server will return a series of sample maps at different zoom levels. If it fails you will get the error report from mapnik, to give you a hint as to what is wrong with your style sheet!
• Update the stylesheet until you are happy with the samples.
• Ask the server to deploy the map - a complete tileset will be created and your map will be visible on the site.

Note that I am not proposing to offer any clever stylesheet editing tools at the moment - that is a possible future extension - the cost of using this service will be learning mapnik XML stylesheet editing.

The Map Renderer

The map renderer computer will do three main tasks:

• Update its OSM database from published diffs (either minutely or less frequent), then re-render all map tiles that have been affected by the changes, and upload them to the web server.
• Watch the web server for 'show me a sample of this file' requests, produce the sample images, and upload them to the server.
• Watch the web server for someone deploying a new map (either changing the mapnik style file, or deploying a new map overlay altogether) - It then re-generates the entire tileset and uploads it to the web server.

Note that this means that the map renderer is pushing data to the web server, rather than the the web server pulling it - This means that the renderer processes can be simple daemons - no need for a server to run on the map renderer, which should reduce security issues (I think!).

Implementation

• Use a commercial web hosting service to host the maps (rather than a virtual server) to keep the costs down.
• To achieve this it will use commonly available tools on commercial web hosts - ie php rather than python scripts on the server.
• Maintain an up-to-date database of UK OSM data on a separate computer, and use this to re-render the map tiles as the map changes. This will initially be my home server, but if enough people are interested in contributing to it, we could change to to a more 'community owned' virtual server that other users can have access to. Note that the only reason I am proposing to limit this to the UK is practicality of data handling - my computer can cope with the UK dataset, but really struggled when I tried to extend it to Europe.
• To create a new map overlay, the map creator will upload their mapnik style file to the tile server. The rendering computer will look at the tile server periodically to see if new maps have been added and render them. A facility will be provided to create a sample at different zoom levels.
• Provide a facility to easily update the map tiles on the host as they are re-rendered.
• Make development of mapnik stylesheets easier by providing a 'produce sample' function to show what the map will look like at different zoom levels.
• Package the software nicely so other people can deploy their own systems easily without having to overcome the learning curve that I have! I envisage providing an Ubuntu package that will set up the renderer computer. A script will be provided that will upload the required files to the web server to give a basic set-up.

The relationship between the various computers in the proposed system is shown below:

Where are we with the Project?

• Hosting - I have a web hosting package with Cloud Next, which is visible at maps3.org.uk. It has a 20GB capacity, so should do for quite a few map overlays. We can upgrade it to an 'unlimited' capacity and bandwidth service for ~£70 per year, which is not bad.
• Rendering - My home server contains the Great Britain OSM extract. I have had it working before comfortably with daily diffs, and have shown that it is capable of using minutely updates (Minutely_Mapnik). I still have a problem with minutely updates growing the database, which needs resolving. It can render tilesets down to zoom level 15 in about an hour, which is fine, using generate_tiles.py from OSM.
• Tileset Updating - at the moment the only way I have of updating the map tiles is to re-generate the lot, which is inefficient - The best way would be to use osm2pgsql's expired tile list, and just render the ones that have changed - this is on the todo list.
• Upload to server - uploading loads of little image files to the server is troublesome and is very slow. Instead tilesets will be packaged into compressed tar archives and uploaded to the server using a php script from the web interface. The archive will be extracted into the correct location on the server.
• Source code archive - I am currently using part of my odds and ends archive at http://code.google.com/p/ntmisc - You can browse the source code here. I will create a separate project for it if it seems useful.

Things to Do

• File Upload facility (better way of uploading and updating tilesets) - work in progress - basic version working, but with no security or checking that you have given it a tileset archive to install. Done!
• Produce a picture showing how the project will be configured - different computers, databases file structure etc. (it is in my head at the moment). Done (roughly - see above.
• Nice Site design (main map, edit pane, 'news' facility, comments) - This is a problem, because I am no designer!
• Get automatic updating of the database and tilesets working = rendering daemon(s).
• Deploy Potlatch2 to allow editing of the specialised maps (ie limit it to just adding specific types of PoIs.
• Work on packaging the system to make it easy for other people to deploy (probably an ubuntu package).
• Get people interested in contributing to the site development and use...

Other Things

I play with a few bits of software, but keep losing interest before they are properly finished. Typical examples are

• wherewasi which is a little analyser for GPX files to see how far you have been and where you were (on an OSM map)
• GPX Route Editor which lets you draw a proposed route on a map (alas not OSM this time - I fell out with OpenLayers and JavaScript and the job stalled), then download it as a route onto your GPS device.
• TownGuide which produces PDF maps from OSM data with a street index and user selectable points of interest highlighted - needs some work on the mapnik style file to improve output resolution for paper output.

See maps.webhop.net for a more complete list.

I also looked after the OpenStreetMap involvement in Google_Summer_of_Code/2010, and will be involved in 2011 too.

Contact

Email grahamjones139 at gmail.com.