Import/GrandJunctionCOUSAFireHydrants

From OpenStreetMap Wiki
Jump to navigation Jump to search

Grand Junction, Colorado, USA Fire Hydrant Import is an import of Fire Hydrant dataset which is of type (data type) covering Grand Junction, Colorado. The import is currently (as of December 2018) at the planning stage.

Goals

Add Grand Junction Fire Hydrants to the OpenStreetMap Database

Schedule

  • 2018-11 Get Permissions
  • 2018-12 Start working on import

Import Data

Data was emailed to me (vorpalblade77) from the Grand Junction GIS Department.

Background

Data source site: https://arcgis-app.gjcity.org/Public%20Safety%20Map%20External/
Data license: http://www.gjcity.org/Maps/
Type of license (if applicable): e.g. CC-BY-SA, Public Domain, Public Domain with Attribution, etc.
Link to permission (if required): e.g. link to mail list reference url - http://lists.openstreetmap.org/pipermail/imports/2012-December/001617.html
OSM attribution (if required): http://wiki.openstreetmap.org/wiki/Contributors#yourdataprovider
ODbL Compliance verified: yes/no

OSM Data Files

https://drive.google.com/open?id=1m90jf5ZT43B8biE0-42C7z8m3slo2yvZ

Import Type

Every fire hydrant has a unique id (HYG_NUMBER) that should be able to be used for conflation purposes after the first import, so while I don't intend to update it every month, it should be easy to update every year (or month, if someone else wants to do it).

I will be using JOSM with the Conflation plugin and the validator.mapcss file that I have included on this page.

Data Preparation

Data Reduction & Simplification

Merge with pre-existing fire hydrants -- there are very few of them, most of them added by me (vorpalblade77).

Tagging Plans

HYG_NUMBER (Hydrant Number) => ref

HT_TEST_DT (Test Date) => check_date, operational_status:date, or survey:date (this would require discussion with other OpenStreetMap users and mappers)

HT_RHP(Residual Pressure) and HT_THSP (Static pressure) => highest becomes fire_hydrant:pressure (maybe add a new tags for difference in static/residual pressures?)

HT_FLOW (Flow Rate) and HT_Q20 (Flow at 20 PSI) => flow_rate (probably use the higher of the two), possibly use fire:hydrant:awwa_class

HY_MDL_NO => model (maybe add manufacturer as well)

HY_SYS_TY => operator (probably change CITY to "City of Grand Junction, CO" or similar)

Changeset Tags

source=City of Grand Junction, Colorado GIS Department

comment=Import of Fire Hydrants from the City of Grand Junction, Colorado

Data Transformation

I'm going to use JOSM with the Conflation plugin and a validator.mapcss file (see Validation below) and convert operator=CITY to operator=City of Grand Junction, Colorado and remove operators=PRIVATE.

Data Transformation Results

https://drive.google.com/open?id=1Jy7LAMTsY4S1OVGkxbDqtD9PtpekYyqg

Data Merge Workflow

Team Approach

This is a solo project.

References

List all factors that will be evaluated in the import.

Workflow

Detail the steps you'll take during the actual import.

Information to include:

  1. Download the original file (shapefiles in zip) and import into JOSM
  2. Add the validator.mapcss file to the validation checks
  3. Run the checks
  4. Check for conflation possibilities
  5. Upload in chunks of 100 fire hydrants at a time.

If needed, this changeset should be easy to revert using the reverter plugin since the nodes are not connected to anything else.

Conflation

I will use an overpass query (see below) to download fire hydrants in the area in question for conflation purposes.

[out:xml][timeout:90][bbox:{{bbox}}];
(
  node["emergency"="fire_hydrant"];
);
(._;>;);
out meta;

QA

Add your QA plan here.

Validation

meta
{
	title: "Validation rules for City of Grand Junction Fire Hydrants";
	version: "2018-12-01";
	description: "Methods to change city data tags to osm data tags";
	author: "Taylor Smock";
}
/**************
 * Conversion *
 **************/
*[HY_MDL_NO][!emergency],
*[fire_hydrant:pressure][emergency!=fire_hydrant] {
	throwError: tr("{0} indicates fire hydrant", "{0.key}");
	fixAdd: "emergency=fire_hydrant";
}
*[HY_MDL_NO] {
	throwError: tr("HY_MDL_NO should be model");
	fixChangeKey: "HY_MDL_NO => model";
}
*[HYG_NUMBER] {
	throwError: tr("HYG_NUMBER should be ref");
	fixChangeKey: "HYG_NUMBER => ref";
}

*[HY_SYS_TY] {
	throwError: tr("HY_SYS_TY should be operator ({0})", tag("HY_SYS_TY"));
}

*[HT_TEST_DT] {
	throwError: tr("HT_TEST_DT should be check_date");
	fixChangeKey: "HT_TEST_DT => check_date";
}

*[HT_Q20] {
	throwError: tr("HT_Q20 should be flow_rate");
	fixChangeKey: "HT_Q20 => flow_rate";
}
*[HT_FLOW] {
	throwError: tr("HT_FLOW is lower than HT_Q20, which matches with AWWA classifications");
	fixRemove: "HT_FLOW";
}

*[HT_RHP][HT_THSP][tag("HT_RHP") >= tag("HT_THSP")] {
	throwError: tr("HT_RHP should be fire_hydrant:pressure");
	fixChangeKey: "HT_RHP => fire_hydrant:pressure";
	fixRemove: "HT_THSP";
}
*[HT_RHP][HT_THSP][tag("HT_RHP") < tag("HT_THSP")] {
	throwError:tr("HT_THSP should be fire_hydrant:pressure");
	fixChangeKey: "HT_THSP => fire_hydrant:pressure";
	fixRemove: "HT_RHP";
}
/**********
 * Checks *
 **********/
*[emergency=fire_hydrant][flow_rate=~/^(\d*.\d*|\d*)$/][inside("US")] {
	throwError: tr("The US flow rate is usually usgal/min");
	fixAdd: concat("flow_rate=", tag("flow_rate"), " usgal/min");
	assertMatch: "node emergency=fire_hydrant flow_rate=24000";
	assertNoMatch: "node emergency=fire_hydrant flow_rate=\"24000 usgal/min\"";
}
*[emergency=fire_hydrant][flow_rate][flow_rate!~/^(\d*|\d*.\d*|.*usgal\/min)$/][inside("US")] {
	throwError: tr("The US flow rate is usgal/min -- please check the units");
	assertMatch: "node emergency=fire_hydrant flow_rate=24000";
	assertMatch: "node emergency=fire_hydrant flow_rate=\"24000 gal/min\"";
	assertNoMatch: "node emergency=fire_hydrant flow_rate=\"24000 usgal/min\"";
	assertNoMatch: "node emergency=fire_hydrant";
}

/**************************
 * Additional Information *
 **************************/

*[emergency=fire_hydrant][!fire_hydrant:type] {
	throwWarning: tr("There should be a fire hydrant type for {0}", tag("model"));
	suggestAlternative: "fire_hydrant:type";
}

/* Models
 * AD57, AD75, AD78, AD80, AD82, AD88, AD89, AD90, AD92, AD99
 * C75, C77, C79, C80, C81, C82, C83, C84, C85, C86, C87, C88
 * C89, C91, C92, C94, C95, C96, C97, C98, C99
 * K00, K01, K02, K04, K05, K06, K07, K08, K09, K10, K11, K12
 * K13, K15, K76, K78, K92
 * M, M00, M01, M02, M03, M04, M05, M06, M07, M08, M09, M10
 * M11, M12, M13, M14, M16, M17, M50, M51, M53, M54, M55, M56
 * M57, M58, M59, M60, M61, M63, M64, M65, M66, M67, M68, M69
 * M70, M71, M72, M74, M75, M76, M77, M78, M79, M81, M82, M83
 * M84, M85, M86, M88, M89, M91, M92, M93, M94, M95, M96, M97
 * M98, M99
 * MH70, MH71
 * N10
 */
*[emergency=fire_hydrant][!fire_hydrant:type][model=~/^(K09|M02|M07|M10|M60|M92|M93)$/] {
	set .fire_hydrant_pillar;
}
*[emergency=fire_hydrant].known_type.fire_hydrant_pillar {
	throwWarning: tr("There should be a fire hydrant type (pipe, pillar, wall, or underground)");
	suggestAlternative: "fire_hydrant:type";
	fixAdd: "fire_hydrant:type=pillar";
}

/* Attempt to find fire hydrants that have a type that do not have an autofix */
*[emergency=fire_hydrant].fire_hydrant_pillar {
	set .known_type;
}
*[emergency=fire_hydrant][fire_hydrant:type]!.known_type {
	throwError: tr("There is a fire hydrant model ({0}) for which we have no autofix", tag("model"));
}

See also

The email to the Imports mailing list was sent on YYYY-MM-DD and can be found in the archives of the mailing list at [1].

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Import/GrandJunctionCOUSAFireHydrants&oldid=1869376"