Cupertino import

From OpenStreetMap Wiki
Jump to navigation Jump to search

This page details a completed import of building footprint and address data for the city of Cupertino, California. The data was requested from the city government GIS department by User:Erjiang.


Licensing and restrictions

The data came with a standard disclaimer that it should not be used for planning purposes and that the city does not guarantee the correctness of the data. When asked about restrictions that would prevent it from being imported into OpenStreetMap, the city replied saying, "Our disclaimer does not restrict you from importing the data to openstreetmap."

Original email about disclaimer:

Follow-up email about restrictions:

Pre-existing data

The vast majority of buildings within Cupertino are unmapped. For the buildings that are drawn, there are only a handful of areas that have addresses, and many of the addresses lack addr:street tags. Where available, pre-existing data will be preferred over the imported data, unless imagery or surveys show that one source is far better.

Import contents

Three shapefiles were provided by the city:

  • Building footprints
  • Building addresses (approx. 16,000)
  • Secondary addresses (apartment unit numbers, suite numbers, etc.)

Building footprints

The building footprints provided by the city are current as of 2015 and seem to have pretty good resolution and line up well with Bing imagery.

The building shapefile also has "Height" tags. I asked the city about it, and they said that the height information is measured in feet and was acquired using LIDAR. The height data is only in whole feet, but otherwise seems to be accurate.

Building addresses

Overall, the address data look very good. Addresses for single-family residences (the majority of the data) are mapped to the building entrances, which should make merging with the building footprints very easy. There are some address points that do not match up perfectly to a building entrance or represent a group of buildings. These will be handled manually.

Secondary addresses

There is no plan to import the secondary addresses.


Tagging

Tags are extracted from the address data and reformatted for OSM conventions. The following tags will be added to the buildings and/or address points:

  • addr:housenumber
  • addr:street
  • addr:postcode (zip or zip+4 where available)
  • building=yes
  • height (height in feet divided by 3.281)

The buildings shapefile comes with "Floors" tags that seems like it should indicate the number of floors in the building, but most buildings have Floors=0. Only a handful of buildings have Floors=1, and there do not seem to be any other values in use.

Process

Tasks will be coordinated using the US OSM Tasking Manager: http://tasks.openstreetmap.us/project/6

  1. Merge addresses with building footprints. (See scripts in Github repo)
  2. Manually import good-looking data one neighborhood at a time to resolve conflicts with existing data.
  3. Survey any suspicious data and import only if it is valid.

Contributor's guide

If you are relatively new to using JOSM or importing data, please look at the Louisville, Kentucky/Building Outlines Import/Contributor Guide page. The guide it describes is for Louisville, but the same process applies here, especially with regards to working with data in JOSM.

How to get help

For questions and concerns about the data, contact user "erjiang" via OSM messages. For cases where you are unsure of a specific building and you cannot survey it yourself, you can use your best judgment and then leave an OSM Note at that location asking for someone to examine it. (See this video guide to adding OSM notes in JOSM). If you have many concerns or are very unsure about a region, it's best to leave a comment in the tasking manager and let someone else look into it.

Create an import account

Use a separate account for importing data, instead of your regular OSM account. A good pattern to follow is: your_username_cupertino_import

Acquire JOSM

You will need the JOSM editor to open the import data files. It runs on Java, so it will run on most computers.

Tasking manager

You can find the project in the US OSM Tasking Manager here: [1]

Log in using your new OSM account, and pick a region.


Click the green "Start Mapping" button, which will lock that region so that other people don't try editing it at the same time.

Opening data in JOSM

There will be a link in the comments that looks like "http://erjiang.github.io/-XXXXX.osm". Copy that URL into your browser to download the file, and then open it in JOSM. You will see the building data for that area in JOSM. (If there is more than one link in the comments, one of them is a mistake and is for another region. Download both and then just delete the one that isn't the region you selected.)

Download OSM map data in JOSM (Ctrl+Shift+Down) and make sure to check the "Download as new layer" box. You will now have at least two layers: one with the to-be-imported data, one with OSM data.

Merging data

This is the part that requires careful inspection.

Some buildings in the import are already mapped in JOSM. We prefer to keep the existing OSM building instead of the import data unless the imported data is much better. However, the imported data has some tags that aren't in the OSM data. You can copy the tags from the import data into the existing OSM building:

  1. Examine tags in both data sets to see if there are any conflicts.
  2. Select building in import data.
  3. Select existing OSM building and paste the tags (Ctrl+Shift+V).
  4. Delete building from import data.

Other things to watch out for:

  • Cut-off buildings on the edges of the data. This data set includes some buildings along the boundaries that are actually outside Cupertino. Many of these buildings were cut off by an imaginary boundary line. Delete any buildings that are not whole.
  • Address points that don't correspond to buildings. There are some address points that are not in a building. These might be because the building was demolished, or that the building has not been built yet. Typically, these address points should not be imported and a note should be left for someone to survey.
  • Crooked lines. Some buildings were drawn sloppily, and corners that should be right angles are not right angles. Fix up these building outlines as needed.
  • Points of Interest. Some address points or buildings match up with a corresponding restaurant or other business. You can look up the business's website to see if these should be merged, but be careful: some businesses only occupy one unit of a multi-tenant building.
  • Intersections with existing features. There may be places where buildings intersect streets or other existing OSM data. Check these against aerial imagery.
  • Bad geometry. This includes duplicate points in a way (validator will find this) or buildings that are self-intersecting.


When you are done with this step, you should still have two separate layers, but they should not have any duplicate elements between them.

Run validator

The JOSM validator can catch many common errors in both the OSM data and the imported data. Deselect any data in the JOSM editor and then click the Validation button in the lower-right corner.

Upload data

Make sure you are logged in with your import account, not your regular OSM account.

First, switch to the OSM data layer and upload any changes you made to the existing OSM data. Then, switch to the import data layer and upload the data there to OSM.

Finally, mark the task as completed in the tasking manager.

Data

The data and scripts for this import can be found at: https://github.com/erjiang/cupertino-addresses

If you do not have Python handy, the finalized output, separated into regions, can be found at: http://erjiang.github.io/ (but see the Process section if you are interested in contributing)

Import users

The username "erjiang_imports" will be used for the import.

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Cupertino_import&oldid=1869264"