Automated edits/ChillyDL

From OpenStreetMap Wiki
Jump to navigation Jump to search

site_type to archaeological_site implementation

concept

In a successful voting on the OSM wiki, it has been decided on December 4th, 2022 to replace the tag `site_type` with the tag `archaeological_site`. 100,000+ objects are affected.

As of January 13th, 2023, 75,000 of the occurrences have already been mechanically re-tagged, converting `site_type=` into `archaeological_site=`. It is not clear – at least to me –, if the lot of the OSM data users and renderers have been informed about this change, and if they have had enough time to adapt. At the same time, 45,000 objects are still tagged `site_type=`. Many are not yet re-tagged, and it is not known if and when they will be. 7000+ objects are tagged both, `site_type=` and `archaeological_site=` with identical values. `site_type=` currently has got 332 values, none of which refer to something other than archaeology.

The implementation of the change in tagging could be improved, it has been noted, see e. g. the in the OSM Germany forum.

In a discussion on the OSM forum, I suggested to

  1. now tag all objects tagged `site_type=` simultaneously with `archaeological_site=` and vice versa.
  2. in July 2023, remove the `site_type=` in all the objects simultaneously tagged `archaeological_site=`.

The scope of the mechanical edit is the full database. To avoid unwanted effects, only objects marked as archaeological sites will be included, and smaller portions of the database will be processed.

In preparation of step 1, all objects of the historic group not tagged as archaeological site have been manually reviewed.

implementation of step 1 in JOSM

  1. run overpass query for objects with tag archaeological_site NOT with tag site_type:
    (
    (nwr["archaeological_site"];
    )->.resultset;
    -
    nwr.resultset["site_type"];
    );
    (._;>;);
    out meta;
  2. load into JOSM, select objects with tag “site_type”, add a dummy tag to all objects, immediately delete the dummy tag again, save as .osm-file
  3. in the file, duplicate lines containing archaeological_site and replace archaeological_site with site_type in the original lines, using these regex in Visual Studio Code:
    Find: (.*)k='archaeological_site'(.*)
    Replace with: $1k='site_type'$2\n$1k='archaeological_site'$2
  4. run overpass query for objects with tags of the historic group with value "archaeological_site" AND tag site_type, but NOT with tag archaeological_site:
    (
    (nwr[~":*historic$"~"archaeological_site"]["site_type"];
    )->.resultset;
    -
    nwr.resultset["archaeological_site"];
    );
    (._;>;);
    out meta;
  5. load into JOSM, select objects with tag “archaeological_site”, add a dummy tag to all objects, immediately delete the dummy tag again, save as .osm-file
  6. in the file, duplicate lines containing site_type and replace site_type with archaeological_site in the original lines, using these regex in Visual Studio Code:
    Find: (.*)k='site_type'(.*)
    Replace with: $1k='site_type'$2\n$1k='archaeological_site'$2
  7. combine both files in JOSM and upload to OSM
  8. check upload result for errors or inconsistancies

alternative implementation of step 1 in level0

  1. run overpass query for objects with tag archaeological_site NOT with tag site_type, load into level0, copy plain text from there.
    (
    (nwr["archaeological_site"];
    )->.resultset;
    -
    nwr.resultset["site_type"];
    );
    (._;>;);
    out meta;
  2. duplicate lines containing archaeological_site and replace archaeological_site with site_type in the original lines, using these regex in Visual Studio Code:
    Find: (.*)archaeological_site =(.*)
    Replace with: $1site_type =$2\n$1archaeological_site =$2
  3. copy back to level0 and upload to OSM
  4. run overpass query for objects with tags of the historic group with value "archaeological_site" AND tag site_type, but NOT with tag archaeological_site, load into level0, copy plain text from there.
    (
    (nwr[~":*historic$"~"archaeological_site"]["site_type"];
    )->.resultset;
    -
    nwr.resultset["archaeological_site"];
    );
    (._;>;);
    out meta;
  5. duplicate lines containing site_type and replace site_type with archaeological_site in the original lines, using these regex in Visual Studio Code:
    Find: (.*)site_type =(.*)
    Replace with: $1archaeological_site =$2\n$1site_type =$2
  6. copy back to level0 and upload to OSM
  7. check upload result for errors or inconsistancies

implementation of step 2

  1. run overpass query to make sure that in all objects in the scope, the value of site_type is (still) identical to that of archaeological site. If not, synchronise manually:
    (
    nwr[~":*historic$"~"archaeological_site"]["site_type"];
    )->.resultset;
    nwr.resultset(if:t["site_type"] != t["archaeological_site"]);
    (._;>;);
    out meta;
  2. run overpass query for objects in the historic group with value archaeological_site AND with tag site_type:
    nwr[~":*historic$"~"archaeological_site"]["site_type"];
    (._;>;);
    out meta;
  3. load into JOSM, delete tag site_type
  4. upload to OSM
  5. check upload result for errors or inconsistancies

status

  • pilot on the German state of Mecklenburg-Vorpommern, done January 17th, 2023
  • pilots on Estonia, Norway and Canada, done February4th, 2023
  • manual review of all objects of the historic group not tagged as archaeological site, done February 3rd, 2023
  • production run of simultanious tagging, done February 5th, 2023
  • information of data consumers, done January 20th, 2023
  • review and manual synchronisation of the simultanious tagging, done July 15th, 2023
  • pilot of removal of depreciated tagging on the German state of Mecklenburg-Vorpommern, done July 15th, 2023
  • removal of depreciated tagging, done July 15th, 2023

values affected

As of January 28th, 2023, 13:00 UMT the following values with 10 or more instances will be affected, if the respective objects are part of the historic=archaeological_site group:

  • fortification
  • megalith
  • tumulus
  • mineral_extraction
  • cliff_dwelling
  • tomb
  • earthwork
  • ruins
  • church
  • building
  • temple
  • burial
  • lime_kiln
  • rock_painting
  • bigstone
  • shell_midden
  • settlement
  • roman_road
  • amphitheatre
  • villa
  • earthworks
  • landwehr
  • mine
  • ridge_and_furrow
  • sacrificial_site
  • heritage
  • bridge
  • theatre
  • rock_art
  • threshing_floor
  • structure
  • cairn
  • station
  • castle
  • industrial
  • mound
  • petroglyph
  • mineral_excavation
  • unknown
  • agora
  • road
  • roman_fort
  • cemetery
  • aqueduct
  • hill_fort
  • tell
  • wall
  • pottery_kiln
  • military_trench
  • kiln
  • place_of_worship
  • camp
  • cave
  • rock_shelter
  • sanctuary
  • kiva
  • oppidum
  • burial_site
  • quarry
  • village
  • gate
  • pictographs
  • necropolis:bigstone
  • burial_ground
  • dam
  • farm
  • lynchets
  • well

For up-to-date details on the values see taginfo


Retagging of Maori Pās

issue

Most of New Zealand’s 2000+ pās were tagged historic=pa, typically using this scheme:
historic = pa
site_type = fortification

while only a few were tagged historic=archaeological_site, typically using this scheme:
historic = archaeological_site
archaeological_site = fortification
fortification_type = pa

The effect was that with all the pās tagged historic=pa, the subordinate tagging with site_type=fortification had never been functional. Also, all these pās have never been rendered e.g. on Carto. Furthermore, the transition from site_type= to archaeological_site= for archaeological sites tagged historic=archaeological_site did not include historic=pa, making pās lose their common tag with other archaeological sites in OSM.

concept and implementation

After discussion and consensus on the OSM forum and the talk-nz mailing list, I made the following changes:

Using this code in Overpass Turbo,

 nwr["historic"="pa"]["site_type"="fortification"];
 (._;>;); out meta;

all ca. 2100 objects tagged:

 historic = pa
 site_type = fortification

were mechanically re-tagged in JOSM to:

 historic = archaeological_site
 archaeological_site = fortification
 fortification_type = pa

The remaining ca. 80 objects tagged historic = pa but NOT site_type = fortification I checked and adjusted manually. In New Zealand, those were pās as well, which I re-tagged accordingly. Half a dozen in the US and Europe seemed to be typos of possibly historic = path, which - making not much sense - I removed.

status

done July 30th, 2023