Amsterdam tree import
Amsterdam tree import is an import of (Bomen) dataset which is of type (MIF/MID or CSV) covering (the municipality of Amsterdam in The Netherlands). The import is currently (as of (2024-02-24)) at the planning stage.
Goals
Schedule
Started in December 2023 with converting tags to tags used within OSM. Asked the Dutch community what they thought of it in January 2024. In February 2024 finished with improving data within the dataset itself (e.g. inconsistencies in tagging, duplicate notes...). Also asked the municipality how and if they wanted to be mentioned as a contributor in February 2024. In March 2024 species:wikidata=* will be added. When the municipality has approved of the import and also when the Dutch community has approved of this import plan, then the import will start. Starting with neighbourhood sized imports.
Import Data
Background
Data source site: https://maps.amsterdam.nl/open_geodata/?k=505
Data license: https://maps.amsterdam.nl/open_geodata/terms.php
Link to permission: https://api.data.amsterdam.nl/v1/docs/datasets/bomen.html
OSM attribution: Contributors#Gemeente Amsterdam
ODbL Compliance verified: yes
At the moment of writing this (2024-02-18) there are about 27,000 trees in the OSM database in the municipality of Amsterdam. 423 of these trees have a genus, 513 have a height and 55 have a plant date. While the dataset of the municipality consists of about 270,000 trees (10× as much!), of which about 265,000 have a genus, about 250,000 have a height and about 240,000 have a plant date. Adding this data would greatly enrich the trees in Amsterdam. (pictures will be added when i can upload pictures)
Of the 27,000 trees already present in OSM about 13,000 are within 2 metres of the trees in the external dataset and about 17,000 within 4 metres. There are also about 2,200 natural=tree_row lines present in Amsterdam (2024-02-25). About 1,400 of these lay within 1 metres of the trees in the dataset and about 1,600 within 2 metres. More on this in Amsterdam tree import#Data Merge Workflow.
Disclaimer: the location of the trees in the external dataset have a deviation 10 cm to 10+ m. After reviewing the data it seems that the median deviation is about 2 m and the mean deviation about 3 m. Which results in a low precision but high accuracy. This will be taken into account when importing (see Amsterdam tree import#Workflow).
Import Type
This will be an import that will be repeated about every time the municipality updates its dataset. To make sure the imported data can be updated multiple tags have been added (e.g. tree:ref=*, source=Gemeente Amsterdam and source_date=*)
This import will be done in JOSM using the JOSM/Plugins/Conflation plugin. This plugin will be used to take care of the merging with the already existent trees.
Data Preparation
Data Reduction & Simplification
Removed duplicate trees from the dataset.
Tagging Plans
Used the wiki about trees as a guideline for converting the tags.
| Key | Value |
|---|---|
| denotation | Taken over from OSM tree |
| description | Taken over from OSM tree |
| diameter:range | stamdiameterklasse as available in the dataset |
| genus | soortnaamTop as available in the dataset (the part between parentheses if present) |
| genus:en | Added according to Tag:natural=tree/List of Genus, Leaf cycle, Leaf type |
| genus:nl | soortnaamTop as available in the dataset (the part not between parentheses if present) |
| height:range | boomhoogteklasseActueel as available in the dataset |
| leaf_cycle | Added according to Tag:natural=tree/List of Genus, Leaf cycle, Leaf type |
| leaf_type | Added according to Tag:natural=tree/List of Genus, Leaf cycle, Leaf type |
| name | Taken over from OSM tree |
| name:en | Taken over from OSM tree |
| name:nl | Taken over from OSM tree |
| natural | tree |
| operator | typeBeheerderPlus as available in the dataset (only partly taken over) |
| owner | typeEigenaarPlus as available in the dataset (only partly taken over) |
| ownership | Derived from typeEigenaarPlus as available in the dataset according to the wiki |
| source | Gemeente Amsterdam |
| source:date | age of dataset at time of import (YYYY-MM-DD) |
| species | soortnaam as available in the dataset (only first two words is the species) |
| species:nl | SoortnaamNL as available in the dataset (only taken over when taxon is not known) |
| species:wikidata | Added according to Tag:natural=tree/List of Species |
| start_date | jaarVanAanleg as available in the dataset |
| taxon | soortnaam as available in the dataset (only taken over when more than species is known) |
| taxon:nl | SoortnaamNL as available in the dataset (only taken over when taxon is known) |
| tree:ref | id as available in the dataset |
| wikidata | Taken over from OSM tree |
| wikipedia | Taken over from OSM tree |
Changeset Tags
| Key | Value |
|---|---|
| comment | Import of trees in Amsterdam |
| type | import |
| source | Gemeente Amsterdam |
| source:date | age of dataset at time of import (YYYY-MM-DD) |
| url | https://wiki.openstreetmap.org/wiki/Amsterdam_tree_import |
Data Transformation
Download the .mif- and .mid-file and use QGIS to convert it to a .geojson-file. Which can then be edited in JOSM to convert the tags and add any other tags. The file with updated tags can then again be edited in QGIS to remove duplicate trees. Now you have a file ready to be imported into OSM.
Data Merge Workflow
Team Approach
This is a solo project. Because of that only the data preparation has already taken a few months. And the import itself will probably also take a few more months.
References
Check if there are notable trees in the area in which the import will take place (e.g. name=* or description=*). And check if there are trees present within 2 or 4 metres (the exact distance has yet to be determined and could even differ between imported areas) from the trees that will be imported.
Workflow
- Download of the OSM trees in the area to be imported
- Check if there are notable trees
- Manually update the notable trees (to make sure their history stays intact)
- Use the JOSM/Plugins/Conflation plugin to update already present trees instead of duplicating them
- Use the BGT icoonvisualisatie layer (and PDOK aerial imagery) to review if trees are in the (approximately) correct place
- Download of the OSM tree rows in the area to be imported
- Select the tree rows found with QGIS (see Amsterdam tree import#Conflation)
- Review if the tree rows will indeed be replaced by the imported trees
- Delete the redundant tree rows
- Check errors
- Upload
- What could be the maximum size of an import? The OSM file is 184 MB at the moment.
- If necessary, the changes will be undone using the JOSM/Plugins/Reverter plugin.
Conflation
Conflation will be done using the earlier mentioned JOSM/Plugins/Conflation. With this plugin the already present trees can be updated/replaced by the trees from the dataset. The trees from OSM and the dataset will be assigned to each other only using distance as a variable (2 or 4 metres, has yet to be decided). There will be no tags used in the conflation because the trees with notable tags will be manually updated.
The existing tree rows will be checked in QGIS if they're at less than 2 metres from trees in the dataset. Those tree rows will be deleted an replaced by individual trees from the dataset.
QA
Anybody has questions? See Talk page
See also
The post to the community forum was sent on 2024-02-10 and can be found here and here.