Humanitarian OSM Team/Working groups/Data Quality Control and Assurance/Problem User Escalation

Problem User Escalation Process visual diagram

Problem User Escalation

This document defines the steps in escalating data quality problems encountered with users (mappers/validators)

HOT Quality Control and Assurance Working Group - First Drafted May 20, 2021

Last Update: October 4, 2021

Context

At times there may be mappers who provide low quality contributions, mappers who are validating poorly, or groups of mappers making large scale errors. This document provides a process for escalating these situations so that HOT can handle the issue first before escalating it to broader groups.

Steps

Low Quality Mapping and Validating

First, mappers/validators who notice the low quality mapping or validating contact the user directly:

1. In the Tasking Manager (TM), comment @username on task(s) that are not mapped or validated properly. Also cc:@PMusername to the Project Manager (PM) to make them aware -- proceed to #2
2. Send the user an OSM direct message and/or comment on one of their changesets in OSM, and confirm that their email address in the Tasking Manager is correct.

This document defines the steps in escalating data quality problems encountered with users (mappers/validators)

If no response, contact the Project Organization and/or the Project Managers/Creator and ask them to address the issue or prevent it from happening:

1. Contact the Project Managers (PMs) on projects the user is mapping, by looking at their changeset comments and hashtags
   1. Ideally directly: the project manager is found in the project’s "Coordination" section
      1. If not, using the project’s "Questions and Comments" section and/or using the #managers hashtag to tag the Project Manager/s
      2. Check if PM is on HOTOSM Slack and message them directly
      3. Use HOTOSM Slack channels: #validation or #mapper_support
   2. Project Managers should then follow these steps
      1. Change the project to include preset restrictions. This is done in the Metadata section of the project. For example, if the mapper is adding bad roads in a buildings-only project, set the project to building=yes only. If they are adding motorways that should be track, allow only certain highway classifications using preset restrictions.
      2. Change "who can map" and/or "who can validate" to a specific team, or only to mappers with intermediate/advanced level
      3. Add a validation team to the project to fix the data
      4. If the project is no longer needed, archive it

If the user is validating poorly, the PM needs to take additional steps:

1. Steps to address low quality validation:
   1. Move tasks that were validated poorly back to the mapped state, or “request revalidation”
   2. Change that project’s settings so that user can no longer validate
   3. Remove the user from the validation team of which they were a part and direct the user to training materials for validation.
   4. If the user is part of the global HOT validators team, request to the Quality Control and Assurance Working gGroup (QC/A WG) that the user be removed from the validators team via the #quality_wg channel in HOTOSM Slack or via email dataquality@hotosm.org.

Organized Editing Issues

If the mapper appears to be part of an organized mapping effort (eg Mapathon), members of the QC/A WG need to attempt to find that group and contact them. The OSMF Organized Editing Guidelines and why we are validating should be emphasized and communicated to them.

1. Attempt to find the organizers of the mapathon group
   1. Check the hashtags the mapper is using and see what they might mean. For example if they add #BigCompany, they are likely part of a mapathon at that company
   2. Usernames may be real names, the user might be on Linkedin or can be found via a web search
   3. Ask HOT Partnerships or Community team if they know about the group (email: partnershipsandcommunity@hotosm.org or via HOTOSM Slack)

Further Escalation

QC/A WG members can also just fix the data, if the person isn’t responding and no other steps are working

1. If user cannot be contacted and there are not too many errors or problems with the data, there is no need to escalate further, just fix the issues

If the user has not responded to multiple modes of inquiry, the QC/A WG will determine if the user should be temporarily blocked from the Tasking Manager.

• Reach out to a TM admin to block the user (hot_tech email: tech@hotosm.org or via HOTOSM Slack @tech)

If user finds ways to circumvent the Tasking Manager block, QC/A WG will contact OSMF Data Working Group (DWG):

1. Only if all else fails: Notify the OSMF Data Working Group and ask them to block the user and if needed revert their changes: data@osmfoundation.org
   1. This should be an absolute last resort
   2. This may take time (DWG are volunteers)