Things To Do
 |
It has been proposed that this page be deleted or replaced by a redirect. (Discuss) |  |
 |
This article or section contains out-of-date information. The information is no longer correct, or no longer has relevance. If you know about the current state of affairs, please help keep everyone informed by updating this information |
This page is aimed at people with programming skills who are (relatively) new to OSM. We will try to give you an overview where we stand on various technology "fronts", what our current issues are, and how you might be able to help.
Note that in addition to the more general tasks listed here, there are always lots of bugs, issues, and enhancement requests for current software in our trac Bugtracker, which you're welcome to work on. See Accounts if you need an account to commit changes to trac.
See also Develop for a collection of developer resources in this Wiki.
See Top Ten Tasks for a high priority which development tasks which will really help the project (but which often require more working knowledge of OSM)
System Overview
Main article "Component overview"
At the (technological) heart of the OSM project is the database server running PostgreSQL, and the Protocol interface implemented in Ruby on Rails; An API through which applications can read and write data. Direct SQL access is not provided, but in regular intervals a Planet.osm dump of the current (not historical) data is created for read-only access.
Maps are usually displayed using the "slippy map" technique, where pre-rendered 256×256 pixel bitmaps are placed alongside each other using JavaScript (OpenLayers framework) to make a nice-looking, zoomable and pannable map. Tiles are rendered for zoom levels 0 (one tile covers the whole world) to 17 (world covered by 417 tiles). We have two major renderers — a fast, database-backed C solution named Mapnik and a distributed, community rendering effort called Tiles@Home which uses a set of XSLTs called Osmarender. Each of them has certain strengths and weaknesses. There are also export scripts for generating Garmin maps or creating PostScript directly, as well as some scripts that do fancy stuff with pre-rendered tiles.
Our two main editors are an offline Java editor named JOSM, for which lots of "third party" plugins are available and which most people are using for intensive data work, and Potlatch, which works on-line and is a very user-friendly Flash application aimed primarily at the casual mapper. There are other editors as well, plus a multitude of import scripts for various data sources.
Project co-ordination and documentation is done by the Wiki you're reading, plus a "trac"/SVN system for source version control and trouble ticket management. In addition to that, we have a number of mailing lists. A large portion of "insider discussion" is also going on IRC, where many core developers hang out. (See Contact)
The project has no programming language preference. All major scripting languages — Perl, Python, Ruby, JavaScript — as well as Java and C are all heavily used. Being "open" we of course lean towards languages that match the philosophy but we're not religious about it. (So if you want to do cool stuff in Visual Basic, go ahead!)
Things To Do with the Main Website
Task: Improve search box
See Search box.
Task: SVG Export
Most people will probably want to edit the exported SVG map from Mapnik using Inkscape. It would be convenient to have the SVG image divided into layers such as "Buildings", "Water", "Forest", "III. class roads" to enable the user to change its style. This requires to change mapnik to put all similar map features into following structure:
<g inkscape:label="Buildings" inkscape:groupmode="layer" id="Buildings"> ... </g>
Additionaly, it would be nice to clip all the SVG paths to fit the selected area.
Task: Exporting to Adobe Illustrator
Add an option to the 'Export' tab to download OpenStreetMap data of a given bounding box as an unstyled Adobe Illustrator file (.ai), see Exporting to Adobe Illustrator.
Task: Homepage improvements for mobile users
The site looks nice when viewed from a smartphone, but could probably be improved upon. The recent OpenLayers mobile sprint should help make the map more easily navigable with touch gestures, but perhaps we should also consider moving the navigation controls so they can be more easily used (e.g. arrows to their respective edge of the map and buttons made easier to touch). We also don't show 'OpenStreetMap' on the pages when they are viewed from a phone, so it's be good to include that without increasing the amount of space needed for the header.
Things To Do with the Data
Task: Data Consistency Checks
The OpenStreetMap data model currently consists of nodes and ways connecting multiple nodes into linear or planar features.
OpenStreetMap does not have strict value and consistency checks for data, which makes many things easier. However, a lot of rubbish accumulates in our database that nobody notices. If you can handle large XML files, you could do a a lot of analyses of the planet file, and find — among other things — ways with unnecessary tags, multiple nodes on the same spot, objects referencing deleted objects, ways consisting of nodes that are only a few metres apart (often created through automatic imports), holes in coastlines, spelling errors in tags, and so on. All these could be determined automatically, but are not at the moment.
We already have some editor plugins (e.g. Validator) and rendering support (Maplint) in place, but these only cover a fraction of what was mentioned above and only make sense when somebody actually works on an area. What we could really use is some comprehensive analysis of the planet file alerting people to "buggy areas", or for some things we could even have automated fixing of things by a robot. Such actions would probably have to be backed by a bit of community decision-making in order not to break something.
Skills required: This task will require some experience in dealing with large amounts of data (you cannot just slurp the planet file into memory and hope to do XPath queries on it — it is too large for that), but other than that is very well suited even for an OSM beginner, and will provide you with a very good idea of what this is all about.
One might start with...
- Write a bot to validate website=xxx tags, perhaps even verifying the name of the node matches the website in some way. Unreachable websites are an excellent indication of an OSM feature which needs to be updated or deleted.
- Find probable duplicate nodes by a fuzzy match (e.g. same name, similar area).
Whom to talk to: For more information/ideas about this task, contact User:Frederik Ramm.
People who are working on this: tenter, Gary68 (Quality Assurance, WayCheck, SomeChecks, osmdiff, OSB Reports, Mapping Quality), e9625163
What country is something in?
Quite a lot of OSM data needs to be interpreted in context. highway=trunk, for example, will have a different default speed limit in the UK than it will in France.
To aid processing of this data, we need a library/webservice that (given a lat/long) uses OSM data to find out, at least, which country it's in. (Ideally, it should cope with other administrative units, e.g. county, city.)
People who are working on this: roland.olbricht
Things To Do with the Database
Task: Satellite Instances
We need the ability to have an offline instance of the database able to replicate (2-way) with the main database.
This is needed for use in remote locations, especially for Humanitarian operations, where connectivity is erratic yet the local service needs to be reliable
Field workers come back to their office with a GPS full of Tracks & Waypoints which they want to quickly dump along with a set of notes. An administrator will use these to create an updated basemap (including Contours) which is then downloaded onto each GPS unit & the waypoints data wiped (since the data is searchable using the POIs in the basemap).
The software should be configured as a Virtual Machine containing PostGIS, Rails port, Potlatch2, Sahana, mkgmap, Osmosis, Mapnik.
We imagine most of the edits to be done using Potlatch2. If *all* edits are done in this way, then simple negative IDs can be used. However if we allow offline editors to be used on this offline instance then we need a more complex scheme to handle this (add the server instance into a hash? Allocate pools of IDs?). We can use an Osmosis plugin to convert our non-std system to the std system in central db if necessary, however we don't want to maintain diffs of clients like JOSM.
- I don't see why which editor you use in the satellite instance makes any difference to anything - either way you are going to wind up with objects created locally whose IDs conflict with the main server and are going to have to resolve those conflicts somehow. TomH 16:16, 23 November 2010 (UTC)
- If using Potlatch2 then there is only 1 level indirection, so the initial IDs can be replaced by the central/final ID after replication. If using JOSM there are 2 levels of indirection & things get messy if the local server changes it's IDs in the middle of a JOSM editing session. FranBoon
- Potlatch does cache downloaded data as well you know - only as long as the applet is running sure, but it is still caching it. I think you will need to maintain a persistent mapping between local and remote IDs and give new objects from the main database a different ID in the mirror. TomH 16:37, 23 November 2010 (UTC)
Replication conflicts should be skipped to allow other edits to proceed - thus allowing just the few
The edits in the main DB can be set as coming from a generic account (e.g. oxfam-haiti)
- This is discouraged - best practice is for all edits to be properly attributed to their author. TomH 16:16, 23 November 2010 (UTC)
- Anything which makes the system more fragile in a situation of high staff turnover & low access to technical support/training is to be highly discouraged. FranBoon
Task: Tree of Areas
Determine all closed areas from the database, then form a tree of areas contained in each other. For example: Europe > Germany > North Rhine Westfalia > Cologne Autotag GPX tracks with this hierarchy.
(hope you don't mind I edited this page directly)
I have written a GMaps API wrapper in PHP for a recent project, which returns structured hierarchical data about political areas and populates a database with it, including id, name, lat/long, post code number etc. There is a standard called xAL (eXtensible Address Language), which has been inclued as XML namespace into KML. So - hmm - I think the names (probably not the lat/long's) are public domain. One other thing is: We should use the name in the main language of the parent country - "Nordrhein-Westfalen" instead of "North Rhine Westfalia". GMaps API results depend on your Browsers language. So it's the best case to use the language of the country. At first we have to discuss the basics, then we (hope I get some helpers) can start to fill the database.
Task: Split multiple GPX tracks
There are areas where I can hardly see the GPX track which I uploaded because the area is covered by hundreds of long straight GPX lines. I suppose they are created by multiple GPX tracks. If I record a track in London and continue later in Paris, there will be a line from London to Paris. A small piece of software should be able to check GPX tracks and if a distance between two points is much bigger than the distances between the neighbour points, the track should be divided.
Things To Do with Map Displays in general
POI layers
The ever-growing list of Map Features inevitably creates a clamour for these details to be rendered on the map. However, rendering absolutely everything is the enemy of good cartography, and will result in a map that looks like a bollocks.
At the same time, there are those who believe the POIs we do show (e.g. pubs) detract from the main purpose of the map and make it unusable in some circumstances.
One answer is for POIs (Points of Interest) not to be included in the default cartography. Instead, they would be available in layers that the user could turn on and off.
Relevant skills:
- OpenLayers (the JavaScript behind our slippy map)
- Rails coding
- Potentially, some PostgresSQL knowledge (should this be integrated with the osm2pgsql/Mapnik cycle?)
- Awareness of performance issues (if done badly, this could utterly boggle any server)
- Some design knowledge might be helpful, but not essential - there are already some good icons in SVN (in particular, the superb svg-twotone set)
Future refinements could include collision avoidance (so two POIs don't appear at the same spot) but this wouldn't be necessary for a first release.
- see already made solution here: http://www.lenz-online.de/osm
Other suggestions
Feature proposal: Renderer (software) which produces an animated map (fly over). Output shall be DirectX/OpenGL or a video file (MPEG, AVI, ...). OpenGL is preferrable for its cross platform appeal.
Feature proposal: A way to navigate a program as proposed above by positioning a handheld device: The view should be as if it was a camera above a map, in that position. (Something written in german about this idea is on this userpage.)
Feature proposal: Speech bubbles like Google Maps, i.e. click on an icon (e.g. church) an you get a bubble with additional information. Probably requires new tags to store that information.
Feature proposal: Search the SRTM data for southeast slopes to create a shading layer. This might be included as bottom layer to maps giving the user an idea about the surface shape.
Feature proposal: Add a scale line on the border of the map, to provide a mean of visually estimate distances
Things To Do with Tiles@Home and osmarender
Osmarender is a renderer based on XSL transformations. An input XML file — containing OSM data — is piped through a complex set of transformations to arrive at an output XML file — containing an SVG vector map. This process requires, in theory, only an XSL transformator and something that can display SVG files. Firefox, for example, does both. So, again in theory, here would be an avenue to directly display OSM data without additional software requirements.
It doesn't work like that in real life for a number of reasons. Firstly, the Firefox implementation of XSLT and of SVG are both not good enough for Osmarender's requirements (nearly everybody uses xmlstarlet/xsltproc as the XSL processor and Inkscape for rendering the SVG files). Secondly, a number of helper programs have evolved around Osmarender which are meanwhile almost a requirement to achieve good-looking maps.
Instead of
- OSM-Data → osmarender → SVG → inkscape → display
we now regularly use the tool chain
- OSM-Data → close-areas.pl → Data-with-closed-areas → osmarender → SVG → lines2curves.pl → SVG-with-Beziercurves → inkscape or Batik → display
Read more about: close-areas.pl * lines2curves.pl
Osmarender is used as a stand-alone renderer as well as an integral part of tiles@home rendering, and for many has become synonym with tiles@home (you'll find people saying there is something wrong with tiles@home when in fact there is something wrong with osmarender, and vice versa).
Task: Osmarender Maintenance Second-in-Command
There is currently only one person in the whole universe of OSM contributors who understands Osmarender (this person is 80n, the original author) to a degree where he can actually make big changes. Most others are reasonably confident in introducing little layout changes, but would never dare to, say, do a complete overhaul of the way Osmarender uses stylesheets, or search for bugs.
One person being able to work with such an important piece of software is not enough — we all have other things to do from time to time, and it would be very good to spread the osmarender knowledge to a few more people who would actually be able to make complex changes to Osmarender when the need arises.
Skills required: You should know XSLT, XSLT and XSLT to do this. Some experience with SVG would also be helpful.
One might start with... You would probably start by familiarizing yourself with the current Osmarender5 from SVN, generate a few maps, play with styles, maybe pick up some bugs (or find things that just don't look right all by yourself), fix them.
Whom to talk to: You should talk to 80n, who may be able to explain things to you, and who probably has some not-yet-finished ideas himself.
There are no takers yet for this task. Click on the "edit" link at the top of this task description and add your Wiki username to the "takers=" field if you want to start working on this task!
Task: Unified Preprocessor
As described above, there's a number of helper programs doing one thing or another before or after Osmarender is run. There is a consensus that (a) there should be as little of these utilities as possible, and (b) any sort of postprocessing is less desirable than preprocessing (because postprocessors need to make assumptions about the kind of SVG data produced by osmarender, creating an unnecessary dependency).
It would be very nice to have one preprocessor that "does it all", to arrive at a toolchain like this
- OSM-Data → preprocesor → polished OSM-Data → osmarender → SVG → inkscape → display
This preprocessor could combine all the extra steps listed above, and could perhaps also take on additional tasks currently handled inside osmarender (notably projection, i.e. the translation of lat/lon values to screen vector units; offloading this from osmarender would allow for a better precision and a choice of projections).
Skills required: This task of course requieres good Osmarender knowledge, and perhaps Perl, because some of the current utilities are done in Perl.
One might start with... Anyone doing this would first have to get a solid understanding of what the various tools do.
Whom to talk to: 80n has written Osmarender and Frollo. Frederik Ramm has done close-areas.pl. Barry Crabtree has done the Bezier Curve hinting.
There are no takers yet for this task. Click on the "edit" link at the top of this task description and add your Wiki username to the "takers=" field if you want to start working on this task!
Task: Getting Text Right
The one thing that is really really ugly about Osmarender maps is the way they handle street names. Because Osmarender itself only creates SVG instructions like "please put this text on this path in this font", and has no feedback about font metrics and so on, it does not know whether a text will fit on the given path, and whether it will overlap with other labels in the vicinity. This creates very ugly results especially in situations where a road has a lot of lanes, or is very short, or consists of many short ways. This is something that really needs to be solved.
A first small step has been made by Robert Hart who created a "beautifier" postprocessor that would try to abbreviate or completely remove street names if they're deemed to be too long for the space available, but that solution is not nearly as comprehensive as would be required.
Ideas for solutions:
- Generate statistics that can be used to predict rendered text sizes
- Inkscape --query-all will tell you the size of all objects. A script could generate an svg file with many short strings in it, and deduce from the output of "inkscape --query-all" the kerning metrics. These metrics could be used by Osmarender to guess how big text will come out.
- Read up on the font file format(s) used, and read inkscape source code to see how kerning is calculated, and write a function that calculates text size the same way inkscape does.
- Use Inkscape's "object to path" feature, and create svg snippets for drawing each character needed. Then change osmarender to use these snippets instead of the text-drawing primitives. This solution is rather involved (requires more changes to osmarender, kerning, etc) but gives more control in the end.
Skills required: A lot of wizardry will probably be required.
One might start with... Start by looking at dense maps and you'll see everything that is wrong here.
Whom to talk to: 80n may have some ideas. Also talk to Robert Hart, look him up in the mailing list archives if there's no wiki page.
People who think this would be good: People who are working on this: gfonsecabr (I am implementing something based on: Frank Wagner and Alexander Wolf, "A Combinatorial Framework for Map Labeling".), Esscue (I'm writing a java application which uses batik to get information about rendered text sizes and bounding boxes. The optimization algorithm used is simulated annealing.)
Task: Make tilesGen.pl Multiprocessor Capable
Currently the only way to get tilesAtHome use a multiprocessor/multicore machine effectively is starting multiple clients on one machine, which is not very efficient, to say the least.
Skills required: Perl knowledge, Perl threading and perhaps locking.
One might start with... Start by trying to run tilesAtHome in multiple instances on one machine and see things break.
Whom to talk to: Deelkar
People who think this would be good: People who are working on this: Jocelyn Jaubert
Task: Tune the color settings for the Osmarender map
The color scheme used for Osmarender should be reviewed and modified.
In particular, there is no need for dark green for forests, where some much lighter color would be appropriate (e. g. similar as is used for parks). On the other hand, there are important areas (e. g. water and buildings), that one can not pass through, although they are drawn with faint colors. Moreover, the tertiary ways have very low contrast, which makes hard reading the map, when printed black and white.
Things To Do with Mapnik
Things To Do with the JOSM editor
JOSM is our Java-based offline editor. JOSM has its own trac and SVN system where you'll find a number of bugs and enhancement requests. Generally, JOSM follows the policy to put as much functionality as possible in plugins, and provides a very clear and open interface making the development of plugins easy.
Things To Do with Potlatch
Potlatch C Extensions
Unlike the rest of the OpenStreetMap API, which uses XML, Potlatch communicates with the server using two binary formats: AMF (Actionscript Message Format), for ways and nodes, and SWF (ShockWave Flash), for GPS tracks.
This has several advantages: reduced bandwidth (they are compact formats), improved performance and simpler coding.
At present, the binary formats are packed by the pure Ruby controllers, amf_controller.rb and swf_controller.rb. But since Ruby's string handling is not the fastest, it may be possible to further improve performance by creating new Ruby extensions in C, which the controllers would use to pack the data.
(Note that there are already AMF and SWF-writing libraries in existence — WebORB for Rails, and Ming, respectively — but both implement a far greater set of features than is needed for OSM, and may also have licence/installation issues in an OSM context.)
Skills required: Knowledge of C and Ruby extensions.
One might start with... The existing code for amf_controller.rb and swf_controller.rb will illustrate what is required.
Whom to talk to: Richard is a Potlatch developer and knows most about the internals of the two formats. Tom Hughes is a sysadmin and has already written a C extension for another part of the site (quadtiles).
People who are working on this: rdittrich
ActionScript 3
Ultimately Potlatch should probably move from its current ActionScript 1 to ActionScript 3 (although haXe might also be a contender...). The rewrite would be a pretty huge task to take on single-handed but might be achievable with several developers working in tandem.
Work has begun on this project in the form of Potlatch 2.0; Please see http://potlatchosm.wordpress.com/2010/03/19/potlatch-2-vector-background-layers/ and http://old.opengeodata.org/2009/11/30/introducing-a-new-osm-editor-potlatch-2/index.html for further details.
Things To Do with other editors
Things To Do with routing
- Generate route altitude profiles using SRTM data --Sjors 08:45, 1 May 2008 (UTC)
- Routable Garmin maps -- this involves mostly reverse engineering the format more completely, then updating Mkgmap
- Implement a faster map-rendering for Traveling Salesman
Community Communications
Beginners' documentation
OSM beginners' documentation is unfocused, cluttered, and outdated. The existing Beginners' Guide wholly concentrates on JOSM, which is to some extent teaching people to run before they can walk, and is arranged in an unfriendly, hard-to-navigate hierarchy which makes it difficult to print or follow. Improved documentation could be written on the wiki but could also be done in conjunction with editor developers, so that it can be presented contextually.
Overall Wiki Maintenance
Main article: WikiProject Cleanup
A lot of people have got involved editing the wiki, and putting it to various uses. That makes it a successful wiki on the one hand, but also rather a dire mess on the other. WikiProject Cleanup is a project to tidy up, and move towards a more coherent organisation (while recognising that a wiki will always be a little bit chaotic)
The wiki way is to "be bold", get stuck in and make the improvements, although some of the major restructuring that is necessary will also have the potential to upset people who have strong ideas of their own.
Please do feel free to "be bold". We don't need to use this page to nominate somebody who is "in charge" of the wiki content. People who make consistently good wiki edits, and show respect for other people's work will gradually become more "in charge" in so far as they gain trust and respect, which will allow them to make more sweeping changes. That's the way a wiki community works. Note that this means you shouldn't dive in and immediately start renaming wiki pages here there and everywhere. (See Wiki Help#Some general guidelines)
Wiki clean-up should also be a collaborative process. Please read the WikiProject Cleanup page, and join in with the related discussions.
You might also gain some kudos for figuring out nifty uses for templates, parser functions and other "state-of-the-art" MediaWiki features to solve some wiki problems. On the other hand features such as templates can be over-used (this page is perhaps an example of that!) While they can be used to good effect, this must be balanced against the consideration that they almost always make wiki editing less simplistic, which is bad for wiki newbies.
If you have ideas for technical improvements to the wiki (beyond the wiki editing which anyone can carry out) e.g. extensions to install, take a look at Wiki#Ideas for technical improvements
Mapping Guidance
Many many questions on the mailing lists are about "how do I map this-and-that". The Wiki does have lots of information about that, mainly on the Map Features page, but often it is out of sync with today's reality, and it does not provide enough guidance. OSM is free and open and all, and everybody can tag anything they want, but most mappers really want to be taken by the hand and told how exactly to map what they see on the ground to achieve the best rendering results — a sort of "best practices", with tips for complex road junctions and all that.
Many people use the Proposed Features page and discussions linked from there to explain how they are currently mapping certain real-world situations, and in fact standards may be created on these pages without every finding their way into the main Map Features page.
So this area is something that definitely needs a hand to sort through it all and tell our new mappers how to get the most out of their work.
Skills required: This task could ideally be done by someone with some mapping experience and/or exposure to new mappers, as he or she will know the important issues first-hand.
One might start with... A first try for complex junctions with pictures got posted on the OSM forum.
Whom to talk to:
There are no takers yet for this task. Click on the "edit" link at the top of this task description and add your Wiki username to the "takers=" field if you want to start working on this task!
Machine Readable Feature List
People are developing tools to make mapping easier. Many of these tools need to know which tags are "allowed" (everything is allowed, strictly speaking, but you will normally want to use the same tags everybody uses), and whether they apply to nodes or ways, and maybe even what their common misspellings are. Currently, all these tools — for example Maplint, the Validator Plugin for JOSM, or the prefabricated JOSM application preferences (dropdown boxes) — have their own lists compiled by the respective authors. It may be worthwile to examine ways to extract a machine-readable list of tags from the Wiki, making the Wiki the authoritative source on commonly used tags and values.
Skills required: This task would probably require considerable MediaWiki experience and maybe also some scripting (a.k.a. Bot), but you do not have to be an OSM insider to go about that.
One might start with... You would best start by identifying and contacting the people currently keeping lists of allowed tags and values, and inquire about their requirements and whether they would be happy to use an auto-generated list from the Wiki.
Update: The maplint test not-in-map_features is auto-generated from the wiki by a Perl script. It might be a good idea to factor out the wiki-parsing from the maplint-xslt-generator into a separate module though.
Whom to talk to:
People who are working on this: See: Machine-readable_Map_Feature_list
