School location import for Bosnia-Giga Maps

From OpenStreetMap Wiki
Jump to navigation Jump to search

Giga School Data Import Plan for Bosnia and Herzegovina

This Giga School Data Import is an OpenStreetMap import of Giga Maps school geolocation dataset (point locations of schools) covering Bosnia and Herzegovina. This import is intended to enhance the completeness and accuracy of the existing Bosnia and Herzegovina schools in OSM. The plan outlined here follows the OSM Import Guidelines (Import/Guidelines - OpenStreetMap Wiki).

How the data was collected

UNICEF Bosnia and Herzegovina developed a method to collect information about school locations and digital resources in the schools. This included checking what kind of internet connection each school has, how reliable it is, and what tech equipment they used, based on national ICT standards. They shared a detailed online survey through all 12 regional Ministries of Education. UNICEF then helped clean and check the data, analyze results, and write the final report to share with the Ministry of Education. To ensure accurate answers, a random selection of schools in each region were visited with on-site confirmation of data. The survey was conducted through SurveyMonkey and included a video guide on how to test internet speed, latency, and location using a mobile app called TestNet, made by the national telecom regulator.

Our School Master Data Overview for Bosnia and Herzegovina

After the school survey was completed, the government of Bosnia and Herzegovina shared an open data set with Giga that includes all schools within the country. In total, the official data set for all schools includes 2,010 schools. Our dataset of the schools has undergone data quality analysis, geospatial checks, and has been cross-referenced with existing schools in OSM. Based on our analyses, the data has been divided into the following subsets of data:

Dataflow for Bosnia and Herzegovina School Location Import

Existing Schools in OSM

1,111 school locations from OpenStreetMap (OSM) 

- There are currently 1,111 school locations in OSM which we have compared with Giga's data to avoid duplication, of which:
   - 227 schools in Giga's dataset are within 50m of an OSM school.
   - 159 schools in Giga's dataset are between 50m and 150m of an existing OSM school.
- The school names in Giga's dataset have been compared to nearby OSM schools to determine if they are the same school. Existing OSM schools will receive additional attributes if they match with Giga's schools.

Confirmed schools in Giga Dataset that are similar to schools in OSM
- We were able to confirm 430 schools that are similar to the schools on OSM
- This is the initial dataset that we shall use for our first import after we transform it and conflate it.
- The final number of schools used for the import may differ from this initial number depending on the transformation and conflation process.

Giga Data

1,285 school locations provided to Giga by the Government with a high level of confidence in the accuracy of the locations 

- These schools have passed basic data quality checks to confirm that they are near existing buildings from other datasets and the there are no other schools in the dataset with duplicate locations.

531 school locations provided by the Government that are in potential duplicate locations (not included in initial import) 

- In this case, one or more schools have identical or very close coordinates
- Further review will be required to confirm that the coordinates are accurate and the schools are co-located

194 school locations that appear to be in uninhabited areas (not included in initial import) 

- These school locations are not supported by existing building datasets (Google Open Buildings, Microsoft Global Buildings, and GHSL Built_Surface) and require closer review for higher confidence in accuracy

BRIEF OVERVIEW OF DATA QUALITY ASSURANCE AND TRANSFORMATION PROCESS

We are using an initial dataset of 430 schools from the Giga dataset that are similar to schools on OSM to add extra attributes including internet connection information to them.

We needed to format the names of the schools so that they would be OSM compliant. Through the help of an OSM Bosnia volunteer we were able to do the following:-

  • Format school names from Uppercase to mixed case → fully capitalized names are not accepted.
  • Translate Cyrillic school names to Latin script → non-Latin scripts will be rejected.
    • Supporting attributes to include non-latin names can be included
  • Remove official government prefixes from the names → e.g., “Javna ustanova” (public institution) cannot be part of the school name.
  • City names or addresses cannot be added to the name of an object → only include them if they are part of the official name.


Conflation Process

We then took the schools through the conflation tool on JOSM and resulted with 97 schools conflated and ready for import into OSM.

The output which is an OSM dataset file is as follows:-

  • Total conflated schools in dataset: 203
    • Successful conflation:
      • All schools have less than 0m distance and a conflation score of 1(which is the highest) from their referenced OSM school
      • They have inherited the attributes of the OSM school
    • When reconflated with the original OSM schools, some conflicts will still appear
    • Conflicts are caused by:
      • The new data that we are introducing, especially school names.
        • We are adding official government school names that have been closely reviewed and formatted with the help of the OSM volunteer from Bosnia.
        • We compared the names already available for those points on OSM with the official Government names from our dataset and believed the official govt names to be more accurate hence replacing them with the ones from our dataset.
      • They are bringing conflicts because they also have not yet been uploaded to OSM.
      • The dataset, as it is, is viable to be uploaded.
    • Geometry type: mostly polygons

Goals

  • Improve Map Coverage: Add all schools from the Giga dataset that are not currently in Bosnia and Herzegovina area of OSM, improving the completeness of educational facilities mapping in the region.
  • Accurate School Locations: Provide verified geographic coordinates for schools, aiding local and global initiatives (like connectivity planning) with up-to-date location data.
  • OSM Integration: Integrate an authoritative dataset into OSM while adhering to community guidelines on licensing, tagging, and data quality.

Schedule

  • Planning & Discussion: (May 2025) – Data preparation and community consultation on the import plan (before execution). This includes presenting the plan on the imports mailing list and OSM forum for feedback.
  • Import Execution: (June 2025) – Begin the bulk import after community approval. The import will be done in stages, expected to conclude by (July 2025). Progress will be monitored and shared periodically.
  • Post-Import Review: After completion, conduct a review to fix any issues and ensure all schools were imported correctly, then report back to the community that the import is finished.

Import Data

  • Data Source: Giga (UNICEF/ITU initiative) school locations dataset for Bosnia and Herzegovina. This dataset contains verified locations of schools across the country as seen in Giga Maps you can download the raw data as a csv file through Giga Maps Api. Each school is represented as a geographic coordinate (point) with associated attributes like name and ID.
  • Data License: Open Database License (ODbL). The data is provided under ODbL 1.0, making it fully compatible with OSM’s licensing requirements like in the following import plan (Address Data Import for Athens-Clarke County - OpenStreetMap Wiki). It has an attribution to OSM and Mapbox, which are initial dependencies of our data, as can be seen in the lower right corner of the Giga Maps interface.
  • Data Format: CSV (Comma-Separated Values) format. The source data is tabular, with columns for:
    • Latitude
    • Longitude
    • Giga School Id
    • School Name (Latin Alphabet)
    • School Name (Cyrillic Alphabet)
    • Country Name
    • School Connectivity Status

This CSV will be converted into OSM XML format for import (see Data Conversion section).

Coverage & Accuracy: The dataset covers all known schools in Bosnia and Herzegovina as of the latest collection date (May-2025). Coordinates are expected to be accurate within a few meters (based on GPS or official data). Any obvious location errors will be corrected during preparation. The list has been filtered to exclude schools already present in OSM to avoid duplicates (see Conflation below).

Data Conversion

The CSV source data will be converted to OSM XML format for processing in OSM tools. Conversion will be done on JOSM that reads the CSV and outputs an .osm file following the OSM XML schema. Key steps in conversion include:

Assigning unique identifiers to each school as needed (or using existing IDs from the dataset) in the OSM XML.

Ensuring each school is represented as a <node> element with the appropriate latitude/longitude and tags (amenity, name, etc.) according to the tagging plan.

Validating the produced OSM XML file to make sure it conforms to the schema and contains no errors (e.g., invalid characters or missing data).

The resulting .osm file(s) will be made available to the OSM community for review before import (for example, via a GitHub repository or the OSM wiki). This ensures transparency and allows others to run validations on the data.

OSM Data Files

The .osm file for import is here: Link.

Import Type

This is a one-time import and will be done semi-automatically through GQIS, R-Studio and JOSM. The method for importing the data will be done by using JOSM and the Open Data and Conflation plugins of JOSM to semi-mechanically add in the school locations of Bosnia and Herzegovina. This will take time but will allow for careful and consistent mapping methods of the school locations.

Tagging Plan

Each school record from the source will be mapped to OSM tags according to established tagging conventions. The following plan describes how source attributes will be translated into OSM key/value pairs:

  • amenity=school: Every imported point will be tagged as a school (amenity=school) to designate it as an educational facility in OSM.
  • name=*: If the dataset provides a school name, it will be added as the name tag. (e.g., name=Example Primary School).
  • ref:giga=*: An external reference ID or code will be provided by Giga for each school, it will be preserved with a ref:giga tag for future reference to the source dataset. (More context is shared below in "Value of adding Giga School IDs as References" section)
  • source=Giga (year): Each feature (or changeset) will include a source tag indicating the data comes from Giga’s school data (with the year of the dataset). This may be applied at the changeset level (preferred) to avoid cluttering individual nodes.
  • internet_access=*: The type of connectivity that the school has or whether it has or not.
  • school:type=*: Specifies if a school is private or public.
  • isced:level=*: Specifies the education level of the school (e.g. primary=1, secondary=1).
  • electricity:=* Specifies the availability of electricity in the school (e.g solar, grid, generator etc).

Rationale: The tagging was decided by matching the source fields to existing OSM tags and using standard tags accepted by the OSM community. We can see the example of this in the following link(Address Data Import for Athens-Clarke County - OpenStreetMap Wiki). Unused or non-standard fields from the CSV (e.g., internal codes or metadata not useful for OSM) are dropped to keep the data clean. This ensures the imported data integrates well with existing OSM data and conventions.

You can see the OSM schema here: Link.

Value of adding Giga School IDs as References

Each Giga school ID is a unique digital identifier that gives a school a kind of “digital passport” – a stable identity that can be recognized across different platforms and our APIs. We are including these Giga IDs in the OSM school data because it creates a clear link between the map entry and Giga’s global database of schools. Community updates to OSM data can provide additional information which can be integrated back into Giga Maps data, and potentially into government datasets as well. With a Giga ID attached, anyone using OSM data can find more details about the school (such as its real-time connectivity status or further information that will be availed in the future) in Giga’s systems. The Giga ID is a constant variable that will stay the same across future multiple versions of our dataset, giving OSM users a simple and reliable reference to our data.

This practice aligns with OSM’s objectives and tagging conventions. OpenStreetMap often uses reference tags (e.g. ref:*) to incorporate official or external IDs, making data linkable across different databases. By adding the Giga school ID as a reference (for example, a ref:giga=* tag), we enhance verifiability and enable powerful consistency checks between OSM and the authoritative Giga dataset.

Data Transformation and Simplification 

Before import, the source data undergoes preparation to make it suitable for OSM: 

Removing Existing OSM Data: The list of schools was cross-checked against current OSM data. Any school already mapped in OSM (matched by location and/or name) was removed from the import list to prevent duplicating data- however, additional attributes will be added to the existing school for a more complete dataset. This focuses the import on genuinely missing features and attributes. 

Field Cleanup: The CSV contains various columns, but not all are needed in OSM. Irrelevant or unsupported fields (e.g., data collection timestamps, unused IDs, or attributes outside OSM’s scope) are omitted. Only the fields necessary for OSM tags (as outlined in the Tagging Plan) are retained as seen here also (example). 

Data Quality Checks: Basic validation is performed on the coordinates (e.g., ensuring no obviously wrong coordinates like 0,0 or out-of-country locations). Schools with missing critical information (like no name or unclear location) might be set aside for manual review rather than bulk import. 

No Geometric Simplification: For schools that already exist in OSM as polygons, official attributes will be added and no changes to the geometry will be made. Thus, no line or polygon simplification is required. 

Any transformations done (such as coordinate format conversion or script-based filtering) will be documented. The final dataset ready for import will closely mirror the source data for transparency, with changes only to remove duplicates and apply correct tagging. 

Conflation  

Conflation – the process of reconciling and merging overlapping data sources (Conflation - OpenStreetMap Wiki) – is a critical part of this import. To avoid conflicts with existing OSM data, the following approach is used: 

Pre-import Matching: As noted, the dataset was filtered against OSM to remove schools that appear to already exist. This was done by comparing school names and locations with OSM data. A conservative approach is taken: if there is any doubt (e.g., slightly different name but same location), those entries are flagged for manual review instead of automatic import. 

Manual Verification: During the import, school entries will be double-checked in the editor (JOSM) against the current OSM map. If a matching school is found nearby, the importer will verify if it’s truly the same school. In cases of overlap, the data will not be added twice; instead, the existing OSM feature may be updated if the import provides new information (subject to community consensus). 

JOSM Conflation Plugin: We plan to use the JOSM Conflation plugin for assistance (example). This tool can match imported points with existing OSM points based on proximity and similarity, helping to identify duplicates automatically. The plugin will highlight potential matches so the contributor can decide to merge or skip those points. This semi-automated conflation will ensure efficiency while keeping a human in the loop for judgment calls. 

No Existing Feature Overwrite: If an existing OSM school has slightly different data (e.g., a different name or location), we will not overwrite it blindly. Such cases will be noted and potentially discussed with local mappers or left for separate review. The import’s goal is to add missing schools, and add additional attribute to existing schools. Any updates to existing features, if needed, will be done carefully and will be documented. 

By following this conflation plan, we aim to integrate the Giga data without creating duplicates or inconsistencies in OSM. This approach ensures that the final map is a harmonized combination of OSM’s current data and the new additions from Giga.

 

QA Plan

Quality assurance is paramount for the success of the import. We will implement several QA measures before, during, and after the import: 

Validation Before Upload: The prepared OSM XML file will be loaded into JOSM (or another OSM editor) to run validations. JOSM’s built-in validator will be used to catch errors such as tag syntax mistakes, missing tags, or potential duplicate nodes. Any issues flagged (e.g., duplicate coordinates or missing names where expected) will be resolved prior to uploading data. 

Test Imports: A small subset of the data (for example, one region or a few points) may be imported as a trial. This test run allows us to see if there are any unforeseen problems with the data or tagging. The test changeset will be shared for community feedback. If issues arise, they can be fixed before the full import. 

Ongoing Manual Review: As the import progresses, Giga will manually spot-check the added points. This includes verifying that each school appears in a sensible location (using satellite imagery or local knowledge if available) and that no obvious mistakes have been introduced (such as schools in the middle of oceans or large clusters indicating potential duplicates). 

Post-Import Tools: After uploading, external QA tools will be employed. For example, OSMCha and Osmose can be used to review the changesets and identify any anomalies introduced by the import. These tools can highlight things like overlapping features, misspelled tags, or other inconsistencies. We will run these checks and address any flagged issues promptly (Quality assurance - OpenStreetMap Wiki). 

Community Feedback: The local OSM community will be encouraged to review the imported schools. They might have local insight to confirm if the school exists and is correctly placed. A mechanism (like a shared map or a public task list) could be set up for mappers to report any problems with specific imported points. Any such feedback will be used to correct the data (e.g., removing a point that was wrongly included, or fixing a name). 

By combining automated validation and human oversight, this QA plan strives to maintain high data quality. The goal is that the imported schools meet the same standard as manually mapped features, with minimal need for post-import corrections.

 

Changeset Policy

All changesets created for this import will be carefully managed to ensure they are of reasonable size and properly documented. The policy for changesets is as follows: 

Size Limit: Each changeset will contain a limited number of school points (for example, no more than (500 nodes per changeset). This keeps changesets small enough to review and, if necessary, revert without difficulty. It also reduces server strain by avoiding one massive upload. 

Changeset Tags: We will include clear tags on every changeset for transparency: 

  • comment: A brief description, e.g. "Importing school locations in (Country) from Giga dataset." This will appear in the changeset comment. 
  • source: The source of the data, e.g. "Giga school data (Year)". 
  • import: yes (to flag that this changeset is an import). 
  • source:url: A URL to the data source or documentation (if available publicly). 
  • source:license: ODbL 1.0 (to denote the data license). 
  • import:page: A link to this OSM Wiki page for the import plan. 

 These tags follow the recommended schema for identifying and tracking imports (Import/Plan Outline - OpenStreetMap Wiki). 

Dedicated Account: The import will be performed using a dedicated OSM user account (e.g., GigaMapsImport or similar) as required by the import guidelines (Import/Guidelines - OpenStreetMap Wiki). This account’s profile will clearly indicate its purpose (Giga school data import) and link back to this wiki page. Using a separate account helps isolate the import changesets and makes oversight easier. 

Changeset Comments: In addition to the tags, the changeset comment will mention that it’s part of the Giga schools import and will include a hashtag or identifier (for example, #GigaSchoolImport). This allows anyone to search and find all changesets related to this project. It also signals to other mappers what this changeset is about, inviting review or questions if needed. 

By adhering to this changeset policy, we ensure that the import is transparent and traceable. Every changeset can be easily identified, examined, or rolled back if needed, and the community can follow along with the import’s progress. 


Revert Plan 

If issues arise during or after the import, a revert plan is in place to undo any problematic changes quickly: 

Identifying Issues: The import lead (Giga team) will closely monitor the changesets and community feedback. If significant problems are found (such as a pattern of incorrect locations or tags), the first step is to halt the import and assess the scope of the issue. 

Scoped Reverts: Because the changesets are kept small and tagged, we can target specific subsets of the import for reversion. For example, if a particular batch (changeset) of points was found to be erroneous, that changeset can be reverted independently using OSM tools. We will use the JOSM Reverter plugin or the osm-revert tool to generate revert files for the affected changesets (osm-revert - OpenStreetMap Wiki). Each changeset has an import tag and identifying comments, which simplifies retrieving it for reversion. 

Revert Execution: Reverting will be done by the import account or a designated experienced mapper. The process involves downloading the changeset to revert, reviewing it (to ensure we don’t remove unrelated fixes), and then uploading the revert. This effectively removes the imported points from OSM. All reverts will be documented, and links to the revert changesets will be added to this wiki page for transparency. 

Communication: If a revert is needed, it will be communicated on the same channels used for the import discussion (for example, the imports mailing list and local community forum). The reasons for the revert will be explained, and any lessons learned will be applied before proceeding further (or before attempting a re-import). 

Fallback Plan: In a worst-case scenario, if multiple changesets need to be reverted (e.g., due to a licensing problem or widespread data error), we are prepared to revert the entire import. Because all changesets carry the #GigaSchoolImport tag and are done by one account, we can query and revert them in bulk if necessary. This ensures that the OSM database can be returned to its pre-import state if the import must be aborted. 

The goal of the revert plan is to minimize any lasting negative impact on OSM. By planning for reverts, we acknowledge that mistakes can happen and ensure we have a clear path to undo them. This plan will hopefully never be needed, but it is ready in case things go wrong. 


Contact 

For any questions or coordination regarding this import, please use the following contact information: 

Import Lead / Data Source Contact: Giga Team – The import is conducted by the Giga initiative (co-led by UNICEF/ITU). Contact the Giga mapping team via email at gigabcn@unicef.org for any inquiries. (This email is monitored by the project coordinators responsible for the data and its integration into OSM.) 

OSM Account: GigaMapsImport – The dedicated OSM user account for this import. You can send a message to this account through the OSM website. The account profile will contain details about the import and links to relevant documentation. 

OSM Community Channels: Discussion of this import is happening on the OSM imports mailing list and the local OSM forum for Bosnia and Herzegovina. You can reply to the forum thread (link to be provided) or the mailing list to reach both the Giga team and local mappers. All feedback and assistance are welcome. 

The Giga team is committed to a transparent and collaborative import process. If you notice any issues with the imported data or have suggestions, please reach out. We will be responsive to the OSM community throughout the project and afterward, ensuring the imported data is maintained and improved as needed. 

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=School_location_import_for_Bosnia-Giga_Maps&oldid=2918591"