|Most of the content on this page is copied from the very excellent Louisville,_Kentucky/Building_Outlines_Import/Contributor_Guide - we are adopting the basic outline of their import process, but there are a few minor differences. If you notice any errors, please report them or even better, just to jump in and fix them.|
So you want to help out with the Dakota County building import? Great! Here's how.
- 1 Overview
- 2 1) Select a task and initialize JOSM
- 3 2) Work on Data in JOSM
- 4 4) Upload Data to OSM
- 5 Validation
- 5.1 Did everything import?
- 5.2 Validation errors and warnings to watch out for
- 5.3 If Your Area is Invalidated
- 5.4 Using geofabrik QA tool
- 5.5 Using osmose QA tool
- 6 How to get help
- 7 More info
Thank you for your interest in helping with the Dakota County building import! Here is a guide for how to contribute.
Install and Configure What You Need
The How to start section contains steps that only need to be completed one time at the beginning. This data import process requires a bit more software setup than normal OSM editing, but we'll try to make it fast and painless.
See our Software Setup Guide.
1) Select a task and initialize JOSM
Now that those preparations are complete, we're ready to start working with the building and address data.
The OSM Tasking Manager is our central tool for coordinating this buildings import project.
Take a task
In order to make sure two people don't work on the same block group at the same time, when you "take" a task the task tracker will mark it as "currently worked on" and not give it to anybody else. Here's how:
- Start with the task manager here: http://tasks.openstreetmap.us/project/88 (Note: you should log in with your
- Now you should be able to browse the map of the county and see what areas are complete and what areas need work. You "take" a task by clicking on a block group on the map.
- Grey areas are still available to be imported. Click on one of these grey areas to see the task for that area.
- Green areas have been uploaded *and* verified (checked by a second set of eyes)
- Yellow areas have been uploaded but not yet verified
- Start with areas that you are familiar with - neighborhoods where you have lived or worked, for example
- If you don't care what block group you will work on, you can click on "Contribute" and then "Take a task at random".
- Click the green "Start Mapping" button to lock the task. Only one person can lock a task at a time, so this signals to other mappers that you are working on this task. The lock on the task will automatically expire in 2 hours.
Open in JOSM
Before proceeding farther, make sure JOSM is running (with remote control enabled, as described above)
- Now you should see a dropdown to start mapping. Select
Edit with JOSM. This will load the pre-existing data from OSM into JOSM in a new layer.
- Turn on satellite imagery layer to give you a better frame of reference. Select Imagery -> Bing from the menu bar. The Mapbox Satellite imagery is also good. If imagery is badly misaligned you can adjust it by right clicking the layer and choosing New Offset - this won't affect the building data, only your visuals in JOSM. The best imagery is from Dakota County - https://gis2.co.dakota.mn.us/arcgis/services/AerialPhotography/2016AirPhotoLeafOff6Inch/ImageServer/WMSServer?request=GetCapabilities&service=WMS
2) Work on Data in JOSM
- A side note: JOSM can be a bit daunting to use at first. This is OSM's advanced offline editor, which is quite powerful and allows us to integrate nicely with the task manager and import data. So it is the best fit for this import process.
This step is the bulk of our work. Now we will use JOSM to review and make any changes or merges to the data before uploading to OSM's database. Until the upload step, no changes are made to the public map, so you can still bail out if you have problems.
General JOSM editing info
If you haven't used JOSM before, there is a nice guide on this wiki that may be helpful. For our purposes JOSM/Basic_editing will contain most of what's needed to get up to speed. And there's another guide here from the folks at learnosm.org. (Of course you can also just learn by trying, we're here to help)
Tips for Editing
- Use a mouse, not a trackpad if you can.
- Control + Click will drag the map (or right click).
- Two finger slide up or down will zoom the map (or mouse wheel).
- Command + Shift + V will paste all copied tags into selected item.
- Go to preferences and remap the Delete key to something else (like Eject) if you are on a keyboard without a number pad.
- Add commonly used menu items to the top quick bar.
- Shift Click selects multiple items at once, Control Click deselects.
- Shift Click to select multiple tags at once, right click a tag for more options (copy selected, copy all, etc).
You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.
Importing and Editing Data
The Dakota County buildings and addresses have been combined into one file for import, found in the "import links:" comment of the task (under the Contribute tab). To open a file in JOSM, just click on its link.
These are the building footprints that are ready to directly import into JOSM. They contain building and address information that you may have to trust or may be able to visually verify by looking at the imagery. These building polygons were digitized in a stereo environment that reduces building lean. The reference imagery you may use to proof this data before load may contain building lean and will appear as if there is a shift in the data. The data is positionally correct.
Areas where buildings exist may need to be conflated to their respective OSM buildings using the JOSM Conflation plugin (JOSM/Plugins/Conflation). In some cases, you may just replace them with the new buildings as they were created in a stereo digitizing environment.
For our purposes we'll be using address points as reference layer in the conflation plugin and the existing buildings will be the subject layer.
Steps to do the conflation:
Copy imported data to OSM layer and validate. You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.
3) Run Validator JOSM tool
JOSM has something called a Validator (click for details) that attempts to automatically find common errors and alert us to fixes before we finish.
- Run JOSM validator and review any errors/warnings it presents. Typically these only come up in very dense areas with lots of buildings. Most will be about duplicate nodes or ways, which are minor and should be automatically fixed.
- May find these warnings: Crossing buildings; Self-intersecting ways; Building inside building
Now fix the Errors automatically with JOSM. Click the Errors folder in the bottom right, and click the Fix button (see image to the right). Then re-validate and there should be no more errors! If there are fix them manually.
If there are still Warnings related to buildings or ways, please fix those manually (right click the warning and click Zoom to see it):
- If a way intersects a building, you can modify the road so it goes around the new building. Reference the satellite imagery.
- If the interior of a building should be cut out (eg, for a courtyard), the problem is usually that there are 2 interior shapes. Left click and drag to select the entire interior shape, the double click the 'building' selection.
- If you have serious doubts about the data's accuracy or suspect a technical glitch, contact the rest of the import team for assistance or a second opinion.
Some errors and warnings are from existing data problems with boundaries, roads, tagging etc. If they were already there and not part of buildings or addresses or roads overlapping buildings, you can ignore them.
4) Upload Data to OSM
You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.
Note: If you changed the existing OSM user data (ie, removed buildings or addresses), upload the changes for this layer first! Then upload the data layer from task manager.
You'll be faced with a dialog listing the modified objects (likely a lot, in the case of our import). It will also ask you for a brief comment and source. We're going to follow a consistent format for these two fields throughout the import, which is important as it allows us to track our work within this rather large chunk of data. It also clearly documents our work so that others in the OSM community to understand how this data got here.
JOSM Upload Changeset Comment and Tags
Make sure you include the following info in changesets to make them more easy to track.
If anything special or out of the ordinary happened, include that in the changeset comment. At the very least, each changset should include this line:
This changeset is part of an import of buildings and addresses for the Dakota County. Full details on the import can be found at https://wiki.openstreetmap.org/wiki/Minnesota/DakotaCounty/Buildings_Import
Set the import=yes tag on every changeset.
Source should always be set to:
Mark complete in task manager
Back in your web browser, there is a third box with a field for comment. This is a good place to put qualitative thoughts on how the data seemed to you, which parts you are more or less confident of, or any other notes you think the next person looking over this section should see first.
Then click Mark task as done. Now the area will show as done on the task tracker (colored in yellow).
After areas are marked as done on the task manager, a second mapper will check it over to see if everything looks right. This is our final QA/QC check for the import, so be sure to only mark a task as valid if you are completely sure that it is done and error-free. Don't validate your own tasks. This will allow some cross-over so at least two people will need to have signed off on a task. Things to look for before marking a task as valid:
Did everything import?
Kind of obvious, but buildings and addresses for the task should have made it into OSM.
Validation errors and warnings to watch out for
crossing ways OR crossing buildings
meaning: This happens whenever two ways intersect. With imported buildings, this can happen when buildings were imported on top of pre-existing ways or if there was funky import data for whatever reason. If one of the ways involved is not a building we imported, then don't worry about it (you can definitely fix if you want, but it's not a blocker).
how to fix:
- Check imagery to see what's going on.
- Sometimes the features are just off and can be fixed by moving them as appropriate.
- If a way actually passes underneath a building then set layer=* or covered=* tags as needed.
- A toll booth building over a highway. The correct way to map this is to split the highway into segments with nodes joined to the building way where it passes under the building. The segment under the building should have the covered=yes tag set. The building should also have the building=roof tag set.
- A creek crossing a building. This often indicates that the creek runs through a culvert underground here. If that is the case, then the creek should be split into segments where it passes underground, and both layer=-1 and tunnel=culvert tags should be set on that segment.
building node within a building way
meaning: There are many variants on this, but happens if a node already existed for a building before the building was imported and now there is a building way and node in the same area.
how to fix: Either merge the tags from the building way and node into the building way, or just leave the node and remove the building tag. If the tag is more specific than building=yes, then set that building value on the building way.
meaning: This might indicates that addresses nodes were have been imported twice by accident.
how to fix: You can just run auto-fix on this.
meaning: This might indicate that buildings were imported twice by accident. If this happened from a duplicate import, there will usually be a ton of them (as many as there are buildings, so at least a few hundred). If the duplicate ways are not buildings, then you can ignore this.
how to fix: Use the JOSM reverter plugin roll back all but one of the duplicate changesets. See the reverter plugin page at JOSM/Plugins/Reverter for instructions on how to do that.
abbreviated street name
meaning: This is a pre-existing error in OSM but we should fix it now since it affects the address we are importing and is important for routing. This occurs when an OSM street name is abbreviated (for example: Lighthouse Landing Dr should be Lighthouse Landing Drive)
how to fix:
- Expand the abbreviate of the name= value on the street way
- Also update any address node or building that has the abbreviated street name.
- Open Edit -> Search tool: search for any object with the incorrect addr:street tag (example search string: "addr:street":"Lighthouse Landing Dr") and have the replace selection radio button selected. This will select all the objects with the former name.
- Then use the Tags window to update all these tags at once by setting the correct expanded value for the "addr:street" tag
If Your Area is Invalidated
When an area is invalidated, it is no longer highlighted in the Task Manager. Find it again and click it to see the comments from the admin. If you want to try and fix the errors, just download the JOSM - there is no need to use the .osm file any more since all that data has been sent to OSM already. When you've addressed their concerns upload just the JOSM layer and set the task as done again.
Using geofabrik QA tool
Using osmose QA tool
(replace username_dcmnbuildings by your user account)
How to get help
- General information on Imports in OSM