Basisregistratie Grootschalige Topografie/Rijnland polder water import

From OpenStreetMap Wiki
Jump to navigation Jump to search

Rijnland polder water import (BGT) is an import of (Basisregistratie Grootschalige Topografie) data-sub-set which covers water areas in the Rijnland- District Water Control Board area (western part The Netherlands). The import is currently (as of (2018-02-28)) at the planning stage.

Goals

  1. To improve mapping of water vs land in polder-areas in Rijnland that consist of water for a substantial part (which is often mistaken for land in OSM)
  2. To correct the large number of errors concerning overlapping land/water areas in the current OSM database
  3. To simplify future changes in water/land coverage

Schedule

Start: March 2018, (sample file imported 2018-03-18 after positive feedback from Dutch OSM-community). Small changes to data preparation after positive feedback regarding sample import. 2018-04-13 start of main import: created main batch of OSM-data and imported second polder. Because import will happen manually, one polder at a time with attention to integration in the database this might take until 2019 to complete.

Import Data

Data source site: https://www.kadaster.nl/bgt

Current OSM standard Carto, Meerpolder, NL

Data supplier to Kadaster for water data in the region: https://www.rijnland.net/overig/english
Data license:http://www.nationaalgeoregister.nl/geonetwork/srv/dut/catalog.search#/metadata/2cb4769c-b56e-48fa-8685-c48f61b9a319?tab=general
Type of license (if applicable): originally CC-BY-4.0, with explicit permission for use in OSM under ODBl, see Contributors#Netherlands
Link to permission (if required): https://forum.openstreetmap.org/viewtopic.php?id=59021
OSM attribution : source=BGT; Rijnland
ODbL Compliance verified: yes

BGT-data (which cover not only water, but all topographical aspects) are centrally published by the national Kadaster, but much of the data is supplied to Kadaster by various (local) authorities. The water data in the Rijnland-area is supplied by the Rijnland District Water Control Board.

Rijnland indicated to want to look at the possibilities for helping in data preparation if permission for import in OSM would be obtained.

OSM Standard Carto Waterways outside Meerpolder NL - Z=13 : water ratio overestimated

Background

The area of the Rijnland District Water Control Board in the western part of the Netherlands lies below sea level for the main part and is sculpted by water and water management over the centuries (see this 400 year old mapversus the inversion of land and water 130 years later in this map for example)- Simpy stated it consists of a main water system ("boezem"), which is interconnecting and includes the main lakes and canals that can be used for transport. Within the "boezem" lie many -approx. 200- "polders", which can be regarded as islands within the boezem, surrounded by a -mostly- grass-covered dike behind which is land (mainly grassland) that lies lower than the "boezem". Within the polders there are many small canals ("sloten", some are navigable for small boats / canoes, most are not), that collect water, which is transported up to the boezem by a bucking ("gemaal") or -in earlier days and still sometimes- windmill. From the boezem, superfluous water is transported up to open water, such as the sea or a river with an open connection to the sea.

In Openstreetmap the water and land in the "boezem" is mapped quite well, for a large part by the grace of the early (2009) 3dShapes-import This was a big improvement at the time, but within the polders this import reached the limits of the available data at the time. Currently many of the small canals in the polder are (a) missing (b) drawn very roughly (c) overlapping with landuse-areas from the 3dShapes import that draw land in places where there is water (d) not joining the land, creating gaps in the map. Improving this proved to be very difficult and labour intensive (almost impossible to do large scale) because of continuous manual splitting and joining of thousands and thousands of nodes. This led to many mappers not improving water features, some just further adding rough water features upon land ignoring database errors and others spending many hours trying to repair damage done by other mappers. These errors also lead to unpredictable results in rendering.

Furthermore (to avoid such problems) small water areas are sometimes drawn by mappers as 2 dimensional ways instead of areas, which might work in areas where canals are far apart, but not in areas with many canals next to each other, because that leads to a grave misrepresentation of the land/water ratio that changes for every renderer and zoom level (example: north of the marker the water is described as areas (ratio is constant when zooming in/ou) and south of the marker only as waterway (in the field land/water ratio is the same as in northern part, but ratio on the map changes when zooming/out)

OSM Standard Carto, waterways outside Meerpolder,Z =19; water ratio underestimated

Since a polder can be regarded as an island in the large boezem water, and no other natural or landuse feature normally crosses the dike, an approach using one multipolygon (landuse=grass) -instead of the many joining grass areas- has already proven useful to solve the errors with overlapping land/water areas and to simply adding/refining additional water features (no need to join each water node with a grass node). For adding / refining water elements the data from the new BGT have already been discussed and used, manually tracking from a WMS layer. This data has proven to be very detailed and precise.

Upon further using this method, the limit of what can be reasonably done manually has been reached and instead of tracing over the data manually an import of the same data is required to continue improving the landuse/natural data in the OSM-database

(also see "References" below)

OSM Data Files

Sample file for data reduction and dissolving with first impression of multipolygon (additional landuses must be added): http://www.openkaart.net/Meerpolder_DISSOLVED.osm

Import Type

One time import: when imported and integrated in the database in a multipolygon, additional changes will only be necessary for landscaping projects (most waterways are a few centuries old) and can be easily done manually by tracing small amounts of data from wms-layer (or additional import of separate shapes),

Legger Rijnland - Meerpolder: actual ratio of land vs water

Method of import in OSM-database: JOSM, with attention to integration in existing database.

Data Preparation

Data Reduction & Simplification

Comparison of current OSM data, original and reduced BGT-data (Imagery: PDOK Beeldmateriaal 2016)

In sample file these steps have reduced the data size to less then one third of the original file size :

-Removing tags from source data not necessary for OSM

-Reduce number of vertices (method for main batch: Douglas-Peucker; set to 0,2; shared boundaries=on; using FME)

-Dissolve polygons:

add adjacent areas together to reduce number of elements/members of multipolyon , but keep the primary waterareas ("the stem of the tree") as seperate elements, apart from the neighouring secundary water areas ("the branches that lead to the stem") because these have different properties and this keeps the number of nodes per element and complexity of shapes within workable proportions

Further details about the technique (especially reducing vertices) will be shared when a more definitive approach has been formulated, and this may evolve from the experience with different polders.

The OSM data file above shows the level of detail we wish to maintain (much more than currently in OSM, but less than the very detailed BGT-data in this region), as not to produce an overload of nodes in OSM and keep future editing easier.

Example of dissolve, joining identical water areas, but keeping primary (red) and secundary (blue) water apart

Despite adding many water areas that currently are not in OSM and improving detail for water areas already on the map, this approach reduces the amount of data by approximately one third in the sample case. This is because of the replacement of many separate adjacent identical grass-areas (that join at a place where actually is a canal). See for example the current OSM water/grass data for the area of the sample file above: http:/www.openkaart.net/Meerpolder-current-grass-water.osm

To keep the large multipolygons workable, the members of the multipolygons will be sorted: outer first (single closed way), then inners: first natural=eater, then landuse=*

See also illustrations:

Detail of dissolving: primary canal (red) is still separate area from secundary canals (blue)


Tagging Plans

natural=water

water=ditch for secondary and canal for primairy. This indicaties both the role in transporting water as possible navigability for canoes (ditch:no / canal: probably)

water_system="polder" (as opposed to the main "boezem" system from every polder is isolated; this indicaties that you can't get here frome the main systen without a lock -almost always missing- or portage)

source=BGT; Rijnland (in element tags instead of changeset tags to allow distinction with current element tags from earlier 3D-shapes import)

operator= (derived form source data [key: Category]: primair = Rijnland; secundair= owner), this allows for distinction in OSM between primary and secondary waters in polders, which van be used in generalization of maps ans is also a good indication of possibilities for canoeing

depth= (prescribed -and maintained- depth_ in meters

---

The source data contains many more keys, but are not always relevant for the use in OSM as a general map, are easily accessible through other means and would require quite a bit of maintenance if you want to be sure if they are still correct after a while.

Only a few polder poldersloten have a name (almost only primary waters), and tagging practice in the Netherlands is to put names on a waterway=* within the canal than on the area, consistent with the Wiki for Tag:waterway=canal

Changeset Tags

type=import

website=https://wiki.openstreetmap.org/wiki/Basisregistratie_Grootschalige_Topografie/Rijnland_medium_scale_water_import

Data Merge Workflow

Team Approach

For now I'm doing this solo, but help with problems that arise and reviewing of results and will be asked in users:Netherlands, where this datasource is currently discussed.

Workflow / Conflation / QA

  • Review if existing water / landuse data for a polder in OSM (a) needs improvement (b) is suitable for this import approach (not so big that overview of an area is lost, such as Haarlemmermeerpolder)
  • When suitable, data for the specific polder will be prepared as described above and further adjusted when necesarry (eg. remove water areas where OSM would show culverts instead of water area, make multipolygon for islands in polder water)
  • Changesets will be limited to the area of one polder at a time, import for a new polder will only be started of the previous one has been completed


  • Download current OSM-data for the polder-area, create multipolygon for polder (landuse=grass), create outer for polder and add to multipolygon
  • Use replace geometry for the new MP-outer and one of (many) overlapping landuse=grass areas form the previous 3D-shapes import to maintain link with earlier import history (for the water elements this is -in general, see exception below- not possible in a practical way since this is not 1 new element but many, and it is not clear how that should relate to the many water existing elements: separation between and number of adjecent areas is not the same)
  • All landuse=grass areas will be removed (since this is now covered by one landuse=grass multipolygon)
  • Review current natural=water areas and waterway=*: if there are substantial and detailed water areas not from the previous import, but areas from a human mapper that cannot be easily included in the new dataset, an acknoledgement will be made on this wiki page to respect the the previous work ( "replace geometry" had been gave practical problems with many and large overlapping old/new geometries, not matching in number and unexpected results when old geometries are attached to other geometries). The history is still visible through http://overpass-api.de/achavi/?changeset= Waterways will be maintained if they propose something besides the natural=water (such as navigable or named waterway), but will be replaced by a natural=water if they don't (good practice rule "one feature, one element")


  • Import new water areas; visual check with high quality areal photograph; adjust where necessary
  • Add new water areas as inner to multipolygon
  • Review and move/join etc. the existing land uses other than landuse=grass; add as inner to multipolygon


  • Validate in JOSM, fix validation errors
  • Upload
  • Check if unexpected rendering occurs, check what the cause is and fix if the cause lies in the data (and not in rendering itself)


  • Load current data for the polder above from OSM
  • Add additional landuse=.. areas for places previously left empty, such as "farmyard" or "residential" (before the multipolygon "grey areas" in the osm standard carto. but now assumed to be grass),
  • Add additional landuse areas to multipolygon is inner, validate, upload, check..


References

Background / history


Discussion of this proposal

Acknowledgements

This import replaced some existing geometries (that might have been correct but incomplete or less detailled, see "Conflation") that couldn't be integrated because of practical circumstances from -amongst others- the following mappers:

wvpd

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Basisregistratie_Grootschalige_Topografie/Rijnland_polder_water_import&oldid=2163701"