User:Pnorman/Import Guidelines

Comments are in italics

A checklist

So you think your city/county/state/country government, a non-profit, or some other organization or person has great data that could be used to improve the quality of OpenStreetMap? Here's a quick checklist to get you started on the right path. Most of these areas are expanded in further sections of this page and on related pages. The first few steps aren't very technical, and any motivated person can begin this process, however eventually highly technical skills and general OSM experience will be required.

1. Gain familiarity with the basics of OpenStreetMap, including editing, such as adding details of your neighbourhood. Consider following the beginners' guide.
2. Contact the relevant local OSM community to make sure you're not heading down a path someone else has tried and to get a sense of how receptive the community is to imports. Check for local user groups, local chapters, and country-specific mailing lists.
3. Check the license of the data. If the license of the data is not compatible with the OpenStreetMap license, you can not use the data.
4. Find out what data layers the organization offers. This might be street centerlines, building outlines, waterways, addresses, etc.
5. Discuss with the community the suitability of each layer for importing. Some data can be readily imported without much difficulty, while others are far more difficult (e.g. street centerlines). Also some are broadly accepted for import, while others haven't had much consensus (e.g. parcel boundaries).
6. Develop with the community a process to convert the data to the OSM XML format. This will include defining how to translate data attributes to OSM tags, and how to properly convert the geometry, including any cleanup and simplification that might be necessary.
7. Develop with the community a process to conflate the converted data with OSM. This might include breaking the data into manageable chunks, and using the wiki or some other method for assigning chunks to users.
8. Develop with the community a quality assurance process.

Types of imports

The requirements that imports are subject to depend on what kind of import it is.

Traditional imports

A traditional import is an import that does not involve verification of each object with imagery or surveys. These imports must follow the all of the legal, technical and documentation requirements.

Some examples of traditional imports are PGS, CanVec, TIGER and most European castradal data.

Imports which involve continual updates to OSM data need to document their update process and include it in the community consultation. Substantial changes to the tagging or the addition of new types of data should be reviewed by the community before they are introduced.

Imports like CanVec where many users will be uploading the data they do not need to not need repeatedly go through the import documentation and community consultation process but there must be adequate documentation to ensure that the imports are done properly. This documentation should be periodically reviewed to make sure it remains effective.

Object by object verification

These are imports where each object created is verified with imagery or surveys. This applies when you are importing areas the size of a block, not areas that cover the entire city.

These imports must follow the legal and technical requirements. The source should be documented on the wiki so other users can find it.

Some examples of this type of import are:

• Where you can see a missing road on the map but don't have it's name, so you import that road
• Where you have no high-resolution imagery and know that there is a commercial area, but not its exact boundaries, so you import them
• Where you have new buildings that aren't in your imagery that you cannot get a good GPS trace of so you import them instead

Some examples of imports that are not of this type are:

• city- or village-wide imports

The best practice is to upload after every few objects you add.

Background layers / tracing

This is when you add a background layer or a WMS/TMS layer like you would imagery and use it as another source. These imports only need to follow the legal requirements.

A source=* tag on the changeset is recommended.

Mandatory part

These parts are minimum requirements. A proposed import that meets these minimum and ignores import recommendations is unlikely to be agreeable to the community and meet the community consultation guidelines.

Legal

Imports must be under a license which allows the OSMF to re-license that data if a new license is chosen through the process in the Contributor Terms. Public domain or attribution-only licenses are examples of these.

Some common public domain licenses are

• Public Domain Dedication and License
• CC0

Some common attribution-only licenses

• CC BY
• Open Government License

• Imports that cannot be relicensed and can only be distributed as CC BY-SA or ODbL must have prior approval from the LWG and be done from an account dedicated to only that import (i.e. not an import account used for other imports).
• The contributions of the user doing the import are covered by the Contributor Terms.

Technical

• dedicated account
• large imports must be discussed with system administrators
• have a plan if your upload is interrupted
• keep changesets self-contained unless you have a clear plan from recovering from an interruption
  • don't upload more than 50k objects at once with JOSM

require sorting uploads to avoid 30k nodes followed by 5k ways which can break easily if someone else is editing?

Documentation

• tag changeset with source=*
• tag changeset with import=yes
• tag changeset with website=* pointing to wiki page
• Appropriate wiki page with links to community discussions

Social

• consult with the local community of each area the import will affect.
  • typically mailing list but sometimes forums or another means.
    • Most lists require subscribing to
  • worldwide imports will require consultation with lots of communities.
  • In areas with a very small community, ask talk@ too?
• consult with imports@ mailing list
  • imports list has more experience with imports and more knowledge of how they can go bad
• must be consensus of both local community and imports@ list that the import is okay
  • do not leave this until the last minute for a complex import.
• If the data source adds new information (e.g. goes from offering just roads data to roads data and rivers data) and you want to import the additional data, you need to re-consult

how to make this work for sources like canvec? should the changes for each canvec version be proposed to talk-ca@ and imports@?

• for imports conducted by multiple people (e.g. canvec) the consultation only has to be done once but the import documentation must make sure that people follow the guidelines

Guidelines

Non-mandatory, but if you ignore them likely to cause problems getting community acceptance

• don't replace user-contributed data with imported data
• ask, how will this help grow the OSM community?
• keep meta-data to a minimum
• simplify data

etc