OSM Tasking Manager/Validating data

From OpenStreetMap Wiki
Jump to: navigation, search
Available languages
English français
Partially mapped & validated task on OSM Tasking Manager
OSM has valuable resources, including the mapping data, and more importantly the volunteers who spend time updating & improving it. Validating attempts to ensure both these resources are maintained at a high standard which is always improving.

There are many people who's first attempts at mapping are as a result of HOT, and they may need help & encouragement.

There are also many people who are better mappers than the validator who may be checking what has been done! It's surprising how much you can learn while looking at someone else’s work.

It is easy for even the most experienced mapper to overlook a feature that requires adding to the data, and new mappers often need a little guidance to help them progress. Many experienced mappers will find it difficult to adapt to mapping vastly different terrain, and will have to spend time exploring the tagging wiki to look up tags like 'wadi', and this even extends to frequently used tags like 'highway=residential' which may appear vastly different in an area of round buildings thatched with straw & accessed from unsurfaced roads.

Who, When & What

Anyone can validate! But, do not validate your own work. Have a look at what needs to be done & if you are confident you understand all that is required, and you know how to correct the problems, then it could be you. More people are needed to carry out validating, but it has to be done properly – have a look at what is listed below & see if you could do some.

The aim is to have 'stage 1' validating taking place at the same time as the initial mapping, so that any amendments can be made while the nature of the terrain, imagery, and features is still fresh in everyone's mind. Prompt validation is likely to lead to the overall task being completed quicker, and any problems can be quickly dealt with. Feedback should be as soon as possible to help & encourage, and possibly prevent the same problem occurring many more times. If everything is working well then the stage 1 validating of a square will take considerably less time than the initial mapping did.

Shouldn't we all be doing all of the things listed in stage 1 validating when we do the initial mapping?

To make it easier to direct a mapper to a learning resource that may help, links are starting to be added after the section. For instance, section on Buildings has a link at the end. When validating you could paste this link into the square you have validated, like this

Help here http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data/Buildings_feedback

Before you start

Make sure you know exactly what the task requires, and have read any wiki pages with more information about the area & tags.

  • You may also wish to add the JOSM Style 'HOT-OSM-Validation' which colours the highways to indicate if they comply with Highway Tag Africa. Instructions on how to load it are at JOSM Style wiki (very easy to do). Makes validating much easier - if you need to see more details & less of the 'Data Layer' in for instance residential areas, remember to toggle backwards & forwards into wireframe mode with Ctrl+w.

Stage 1. At neighborhood/TM task scale

Validating 4.png
This process is described for JOSM users, which includes a 'validator'. It is possible to validate using other programmes & tools – if you are an expert in these please add details below.
  • Using the Tasking Manager (TM), load a square into JOSM.

Initial Scan

Have an initial quick scan around to see if you will be continuing or will be invalidating the square. If it is obvious that major improvements need to be made, click on the 'invalidate' button & add a comment explaining what needs to be done. The most frequent problem is a simple one that anyone can make, and the square has somehow been marked as complete, when it is obvious that the whole or majority of the square is still to be mapped – this type of error does not really need a message to the mapper, and may even be a software glitch (That's what we all claim). Do not invalidate a square for something that is not a requirement of the task. You can still invalidate a task later if you need to.

  • Validation results
    If your initial inspection is okay, then Run the Validator and correct everything that can be corrected. These are often things like crossing buildings or ways, but it can get a little more complex on occasions.

Buildings

Validating buildings missing.png
  • Now check if buildings are missing. Not all buildings are rectangular - see the image which contains many round 'building=hut' If it actually represents a consequent number of buildings over a TM task, it can be invalidated. If it's only a few then it's simpler to add them as you check round - with the density of habitations in some areas everyone will miss a building or two at some point.
  • Are the buildings correctly traced. Sometimes mappers do not trace buildings accurately & if significant this needs correcting.
  • More frequent are mappers that do not know how to square buildings. In this case, after having checked what is their preferred editor, it is best to send a message to their OSM message box explaining how to quickly square a building.

Paste the following into a message for the mapper concerned: Help here http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data/Buildings_feedback

Highways

  • Highway tags are correct? Some mappers tag every highway as a track because they are not paved, but this is not a meaningful criteria in these developing countries considering 99% of roads are unpaved. Check the wiki definitions, perhaps the Highway_Tag_Africa is relevant & you should research there before going any further.
  • Road geometry.
    Road Geometry
    Some mappers do not put enough details and others too much (eg a node every 10 or 20 m even if the road is straight). First case is quickly corrected with the (magical) Improve Way Accuracy mode in JOSM; second case requires deleting extra nodes when they actually make a straight road weave.
  • start/end of roads. Some mappers produce long winding ways which meander through several junctions. Click on the 'way' and break it into more obvious parts, probably at junctions or changes in the tagging. Some other streets or roads are broken up without any reason (same tags for all the sections), and need to be combined.
  • See Stage 2 validation re further problems with Highways, some of which may be visible within the single square you are validating.

General issues

Validating changeset missing 1.png
  • Anything else you can see which will make the Stage 2 validating quicker & easier (or it could be very time consuming to clear all the small problems!)
  • When you are satisfied your loaded square is of a high enough standard, upload any changes & press the 'validate' button on TM. Add a comment in the validation box, such as "All complete, thanks. Couple of minor errors with crossing buildings highlighted by the validator programme & I've fixed them", or whatever is relevant. You may also send a message to the mapper concerned, but it is worth looking at a few squares of theirs before doing so. You will find that many of the mappers are new to OSM, and it is worth letting them know that their work was satisfactory, and it would be appreciated if they did some more! (See below for some ideas of what you could include in a message to mappers).

Stage 2. Town or city scale

  • Load as much into JOSM as you are able - you may have to do a larger town or city in stages.
  • Run the Validator and correct everything that can be. (If the validators from stage 1 have being working well, this should not take long!).
  • As you are now looking at a much larger area, it should be apparent which sections of highway should be joined together and tagged as Primary, Secondary, Tertiary etc.. They are often larger and frame a larger area or can be a parallel way to main roads. It is also important to check where they start and when they stop, which is often not possible to do when you are looking at individual squares in the Tasking Manager.

Missing parts of Highways

Looks weird on Mapnik and a check of the imagery confirms the two sides of the road separated by a drain are not finished

Validating 1.png

Road continuity

The situation here seems weird as well as we expect the 2 unclassified roads to be connected and not joined by a simple path. The imagery confirms that the southern highway looks the same, and should be tagged the same, whatever the tag. The example is actually good as farther south it changes for tertiary. No reason for this, says the imagery. It should be cut when it becomes a straight road, though.

Validating 6.png

Isolated sections of Highways

Drivable highways cannot be isolated and connected to the drivable road network by paths, they must be connected to it by driveable highways.

Validating 7.png

Tagging coherence

In this example, the primary and tertiary road are connected by an unclassified road. Same thing between the two tertiary roads. After having checked the imagery this road would deserve to be tagged as tertiary. This obvious example apart, this requires to check the streets width to identify the main ones that needs not to be tagged as residential but unclassified or even tertiary.

Validating 8.png

Tasks awaiting Stage 2 Validation

Useful Tools for validating

Main list is at Quality_Assurance

  • Overpass_turbo
  • Osmose Web browser based. Whole world coverage. Multilingual
  • Keep_right Web browser based. Whole world coverage. Multilingual
  • That_Shouldnt_Be_Possible Web browser based. Requires a gps trace of the area / route you wish to check. At present only limited parts of the world are covered.
  • Visualizing OSM data with cartodb to aid HOTOSM validation. Please see this blogpost

Messages to Mappers

You don't have to send a message to the mapper involved, but if you are going to, this section may help. If your message concerns a problem with the mapping, see the links which are being developed for adding to your message - Problems with Buildings is the first. Of course, you may send your own message to any mapper, but this is a template of a message being sent by Tallguy which may give you some ideas about what you could include.

"Hi & thank you for helping the Humanitarian OpenStreetMap Team (HOT).

I am an experienced mapper (still learning though!), and have looked through one of the squares you have completed for task from the OSM Tasking manager. You got it right, thanks!

If you are a regular contributor to HOT then keep up the good work.

If this is one of your first contributions, then thank you & please keep up the good work. There are many tasks outstanding in http://tasks.hotosm.org/ , all are for very good causes & it would be great if you could keep contributing.

You could also help HOT with more than just mapping. Here are a few ideas; Validating – there's a section on this in http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager Perhaps if we each validated 3 squares and mapped one? We could learn a lot from others & help to raise the standard of mapping throughout. Sometimes you map a square & think “Why doesn't that highway come into my square from that square – when you've finished your square take a moment to have a look – perhaps it needs a lot more work & shouldn’t be left showing as complete? You could also get involved in Stage 2 validation?

Take a moment to read http://learnosm.org/en/ which contains sections on advanced mapping. After 5 years of mapping I read it & still learnt quite a lot,

Help with translating; http://learnosm.org/en/ is multilingual (English | Bahasa Indonesia | 日本語 | Português | Italiano | Français | Deutsch | Español ) but needs more translated especially on the advanced sections for JOSM, http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager (either edit the original, or add to Talk section, or send to Tallguy for adding in – good screenshots are also a help),

Software writers. Could you help the team with https://github.com/hotosm/osm-tasking-manager

http://hot.openstreetmap.org/ and http://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Team will tell you more about HOT. You can keep up to date about HOT, or particular areas by subscribing to the relevant lists at https://lists.openstreetmap.org/listinfo . There is also irc://irc.oftc.net #hot for a link to others mapping for HOT, and the page http://wiki.openstreetmap.org/wiki/Irc lists all of the IRC channels if you have an interest in a particular area.

Keep up the good work & I hope to see more squares completed by you in the future.

Regards

Tallguy"