Import/Catalogue/CoREStack to OSM

From OpenStreetMap Wiki
Jump to navigation Jump to search

This page outlines the plan for importing geospatial data from the CoRE (Commoning for Resilience and Equity) Stack into OpenStreetMap. The CoRE Stack provides a comprehensive digital commons for natural resource management, and this import aims to integrate its multi-layered datasets, including hydrological units into the OSM ecosystem.

This project is currently in the proposal phase and is documented here for community review and feedback in accordance with the OSM Import Guidelines.

exclamation mark icon

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

Goals

The primary goal of this import is to integrate hydrological spatial units from the CoRE (Commoning for Resilience and Equity) Stack into OpenStreetMap (OSM), thereby enhancing the availability of structured, high-quality watershed data in a globally accessible platform.

Specifically, this project aims to:

  • Bridge the gap between analytics-focused geospatial systems and open mapping platforms by translating CoRE’s micro-watershed datasets into OSM-compatible primitives.
  • Enable a globally queryable, topology-aware spatial registry for hydrological units, which is currently not provided by the CoRE Stack.
  • Improve the representation of watershed boundaries in OSM, particularly in under-mapped regions such as rural India.
  • Develop and demonstrate a scalable workflow for data transformation, validation, and redundancy-aware conflation for large-scale geospatial imports.
  • Contribute to the OpenStreetMap community by adding meaningful, feature-specific open geospatial data that can be freely used, validated, and improved by others.

This import is designed to align with OSM’s principles of openness, verifiability, and community collaboration.

Schedule

The project will follow a phased timeline as outlined below:

Phase Timeline Status
Familiarization with OSM and sample imports Completed ✔ Done
Data collection and pre-processing By April 2026 In Progress
Community discussion By 15 April 2026 Pending
Test import (after approval) By 25 April 2026 Planned
Full import pipeline By mid-May 2026 Planned

The schedule is subject to adjustments based on community feedback and approval during the discussion phase.

Import Data

Background

The dataset used for this import is derived from the CoRE (Commoning for Resilience and Equity) Stack, which manages large-scale geospatial datasets structured around hydrological units such as micro-watersheds.

OSM Data Files

The processed data files prepared for import include:

  • Cleaned and validated GeoJSON files
  • Converted OSM XML (.osm) files for upload

Data files will be made publicly available at:

Import Type

  • Import type: One-time import(test import and final import)
  • Execution: Automated import using scripts, combined with manual validation

The import will be performed using the OpenStreetMap API in controlled batches(reviewable changsets) to ensure data quality and ease of review.

Tools and methods:

  • Automated scripts for data transformation and upload
  • OpenStreetMap API for batch uploads
  • Manual verification using JOSM before and after upload

Data Preparation

Data Reduction & Simplification

To optimize the dataset for efficient storage and rendering in OpenStreetMap, geometry simplification is applied to all watershed polygons prior to import.

Approach

  • Polygon geometries are simplified using the Python library Shapely.
  • A topology-preserving simplification method is used to ensure that:
    • polygon validity is maintained
    • boundaries do not self-intersect
    • spatial relationships are preserved

Implementation

The simplification workflow involves:

  • Parsing GeoJSON input data
  • Applying simplification to each feature geometry
  • Replacing original geometries with simplified versions
  • Exporting the processed dataset as GeoJSON for further conversion
  • Reference implementation: GitHub: OSM_corestack/scripts

Validation

  • Simplified geometries are validated using JOSM validator tools
  • Visual inspection is performed to ensure no significant loss of detail

Tagging Plans

The imported features represent hydrological units (micro-watersheds) from the CoRE Stack and will be mapped to OSM boundary relations with appropriate tagging.

Core OSM Tags

The following standard OSM tags will be applied:

Tag Description
type=boundary Defines the feature as a boundary relation
source=https://core-stack.org/ Source of the dataset
created_by= creator of that changeset Indicates automated import process
wikipedia=* Link to corresponding Wikipedia page that we have populated with geographical data of past and current year(if available)

Micro-watershed Pages: These pages provide specific hydrological metadata, including annual precipitation, surface runoff, evapotranspiration and net water balance for the current year.

Watershed Pages: These serve as directory pages, containing a structured list of all micro-watershed pages contained within that specific watershed boundary.

Example Documentation Pages

To demonstrate the utility and transparency of the linked data, the following are sample wiki pages that represent the target documentation structure:

  • Microwatershed Data Page (Sample): Example Microwatershed
    • Features: Current year precipitation (mm), estimated surface runoff, evapotranspiration and calculated net water balance.
  • Watershed Directory Page (Sample): Example Watershed
    • Features: A complete spatial registry and directory of the contained microwatersheds within the watershed - C2AGAN72 boundary.

Optional Tags

The website=* tag may be used where an official reference page for the entity exists. Its usage will be limited and applied only when appropriate and verifiable.

CoRE-Specific Tags

The following additional tags are introduced to preserve important metadata from the CoRE dataset:

Tag Description
core_entity=* Type of hydrological unit (e.g., watershed, microwatershed, riverbasin)
core_id=* Unique identifier from CoRE Stack (e.g., 12_305607)
core_updated=* Version or update status (e.g., version1, version2)

These tags are namespaced (core:* equivalent concept) to avoid conflicts with existing OSM tagging conventions and to ensure clarity of origin.

Geometry Representation

  • Features will be represented as multipolygon relations where applicable.
  • Each watershed will be a closed polygon with proper topology.
  • Relations will follow OSM multipolygon standards.

Metadata Handling

  • Timestamps are automatically managed by OSM and will not be explicitly added.
  • Changeset-level metadata will include source and import references.

Tagging Validation

  • Tags will be validated using JOSM prior to upload.
  • Community feedback will be incorporated, especially regarding introduction of CoRE-specific tags.

Data Transformation

The detailed data transformation pipeline, including scripts, tools, and processing steps used to convert CoRE Stack datasets into OSM-compatible formats, will be documented and updated in this section.

This will include:

  • Conversion workflows (GeoJSON → OSM XML)
  • Tag mapping implementation
  • Automation scripts used for batch processing
  • Validation steps during transformation

To be updated.

Data Transformation Results

The outputs of the data transformation process, including processed datasets and OSM-compatible files, will be made publicly available.

These will include:

  • Final GeoJSON files
  • OSM XML (.osm) files prepared for import
  • Sample datasets for review

Data files and results will be hosted at: https://github.com/lmaokisnotcute-4-lbhai/OSM_corestack/tree/main/data

Data Merge Workflow

Team Approach

This import is being conducted as a collaborative project by:

  • Devanshi Malik (Undergraduate student, IIT Delhi)
  • Sweety Kumar (Undergraduate student, IIT Delhi)

The project is carried out under the guidance of Professor Aaditeshwar Seth as part of work involving the CoRE (Commoning for Resilience and Equity) Stack.

The team is responsible for data preparation, validation, and execution of the import, along with incorporating feedback from the OpenStreetMap community.

Workflow

The import will follow a controlled, step-by-step process to ensure data quality and compliance with OSM guidelines:

  1. Load processed OSM-compatible data into JOSM
  2. Download existing OSM data for the target region (Sunderpahari, Jharkhand)
  3. Perform visual inspection and validation of geometries
  4. Check for conflicts with existing OSM features
  5. Apply tagging schema and verify attribute consistency
  6. Upload data using the OSM API in small batches
  7. Review uploaded data on the OSM map and correct any issues

Changeset Size Policy

  • Each changeset will contain a limited number of features (approximately 50–100 polygons).
  • This ensures:
    • easier review by the community
    • reduced risk of large-scale errors
    • better traceability

Revert Plan

  • (to be discussed)

Conflation

  • (to be discussed)

QA

  • Feedback from the OSM community will be incorporated before and during the import process.

Discussion

The import plan will be shared with the OpenStreetMap community for review and feedback.

  • Community discussion post date: (to be updated)
  • Discussion link: (to be added)

The import will proceed only after incorporating community feedback and receiving approval.