User:Dalep
About
I am a new OpenStreetMap user starting in mid 2008. I live in Orlando, fl and had originally intend to do some street cleanup of the tiger data and duplicate streets in the area. After seeing how bad some of the 2007 tiger data was I started to look into other data sources, and how to import them. Note: the new 2008 Tiger data is much improved, and when it is incorporated should resolve a lot of issues of accuracy.
Import Work
I have been working to understand and use the tools available to convert free Public Domain GIS data into OSM format and get it imported and merged into the database.
Working with scripts
I have managed to cobble together a working system with perl and python scripts to get shape file data into OSM format. I have now also been able to split the data in smaller usable chunks using osmosis. Progress, but still very labor intensive. I have now also taken some data into JOSM or Merkaartor for final editing and tagging, then uploading.
tools and comands
I am using windows, but most tools are ported from linux or cross platform anyhow. Replace %shape_file% with the actual file names. Also paths may be an issue you will have to deal with for your setup.
Convert Datum
I use ogr2ogr.exe bundled as part of a FWTools package. Both the scripts below CAN handle datum translations, but I prefer to use the PRJ files rather than keep editing the script to match the datums needed.
ogr2ogr.exe -t_srs WGS84 -s_srs "ESRI::%shape_file%.prj" "%shape_file%-out.shp" "%shape_file%.shp"
Only works if there is an included projection file, but you can use a PRJ file from another shape file that matches up if you have it. IE. city-road.prj in place of a missing city-lakes.prj as long as both layers line up when loaded in qgis
Convert to OSM XML format
I know of Several ways that work for different situations. See Shp2osm
Shp-to-osm.jar
A nice fast, easy to use java based app. An straight forward rules.txt file controls how elements are converted/tagged. This approach lacks the capabilities of the scripted versions above, but GREATLY simplifies the process for basic shp files. This also handles the projection conversion seamlessly as long as there is a .prj file. This is now my preferred method for SHP file conversion. See also Shp-to-osm/tiger-rules.txt and NHD_Rules
perl shp2osm.pl
shp2osm.pl source Converts all metadata into OSM tags. This is fine for things like lakes or node data that will be retagged and cleaned up in JOSM
perl.exe shp2osm.pl "%shape_file%-out" > "%shape_file%.osm"
python shape_to_osm.py
From shape_to_osm.py folder Examples are also there to help figure out the script. This script is designed to convert the various metadata into proper OSM tags. As such, it needs editing so that it matches the shape file metadata. Not fun unless you know and like python. It should not be too hard for anyone with scripting or programming experience to see what is being done and modify it to suit your data.
python.exe shape_to_osm.py %1
polyshp2osm.py
I have found but not used polyshp2osm.py, at least not yet. This is a result of the MassGis Layer Openspace import.
Another copy of shp2osm.pl can be found in the same SVN folder as well.
Split files
I'm still refining this, but it is functional for manual imports into OSM. This is pretty slow with larger files (over 1MB)
Split the state into a county based on a bounding polygon. The poly isn't avaliable, so I pull it from the OSM data based on the way name field in this case.
osmosis.bat --rx enableDateParsing=no file="florida.osm" --wkv keyValueList="name.Orange County" --used-node --wx file="orange_b.osm"
Edit it in JOSM to clean it up, and retag the file for use with osm2poly.pl Then use the script to convert it to the poly format (text node list) that osmosis can use.
osm2poly.pl orange_b.osm > orange_b.poly
Use osmosis to pull out the county of data and writhe it to a new file.
osmosis.bat --rx enableDateParsing=no file="florida.osm" --bp file="orange_b.poly" completeWays=yes --wx file="orange.osm"
Then since that is still to unmanagable in JOSM I split it further via a bounding box. In this case just the sides that need to split data are listed instead of all 4 per box.
osmosis.bat --rx enableDateParsing=no file="orange.osm" --tee 4 --bounding-box left=-81.45 top=28.51 completeWays=yes --write-xml orangeSE.osm --bounding-box left=-81.45 bottom=28.51 completeWays=yes --write-xml orangeNE.osm --bounding-box right=-81.45 top=28.51 completeWays=yes --write-xml orangeSW.osm --bounding-box right=-81.45 bottom=28.51 completeWays=yes --write-xml orangeNW.osm
Alternate manual method that involves spliting the data as shape files.
Open file in Qgis and make a selection, either a box, or via the attributes table. Toggle off rendering (makes the process a LOT faster) Right click on the layer, and save selection to new shape file. Toggle editing on and delete selection. Toggle editing off. Select save (have original backed up elsewhere!) Toggle on rendering so you can now see what was removed. Rinse repeat.
Imported data
Hydro stuff, considered temporary until the NHD set is imported, then these can be wiped out.
United States, Florida (Orange county) "source=FDOT:lakes100k:2008-11-14" Orange county only from lakes_with_names.shp 100k hydro data Filling in some missing roads with tiger 2008 data "source=Tiger2008 by DaleP 2009-02-28" if it is totally new or was bad placeholder tiger data before.
United States, Florida (Hillsborough county) "source=Tiger2008 by DaleP 2009-02-28" Hillsborough county Tiger 2008 hydro layer (minus MTFCC:H2053 elements) from tl_2008_12057_areawater.shp Removed due to a bad upload- will re-upload soon
Wiki Contributions
 |
All my contributions to OpenStreetMap are released into the public domain. This applies worldwide. In case this is not legally possible, I grant anyone the right to use my contributions for any purpose, without any conditions, unless such conditions are required by law. |