Import/Catalogue/Recycling Import Norway
Goals
The goals of this import are to:
- Add all recycle centers and containers from Sortere.no to OSM, providing approx. 6.000 new recycling locations in addition to the approx. 1.000 recycling locations already in OSM.
- Add recycling types to each location
Schedule
- 22 March 2018 - Explicit permission and data obtained from Loop/Sortere.no.
- 3 April 2018 - Import plan discussed at the Norwegian mailing list - Link
- 6 April 2018 - Import plan submitted to the Import mailing list.
- 15 April 2018 - Dataset uploaded to OSM - Map on Gjenvinning (Norwegian)
Import data
Background
The various regional recycling companies in Norway have establish Loop, a foundation for promoting recycling. The Norwegian Directorate for Environment Protection is co-funding the activities. The foundation has established a joint database for all recycling sites which is being updated regularly by its member companies. The database is presented on the web site sortere.no and it is available to app developers and map services. OSM is the first organisation to receive the opportunity to use the dataset as open data without any written agreement.
Data source site: Dataset provided from data.sortere.no/api/docs/
Data license: No license defined
Type of license (if applicable): None
Link to permission (if required): Explicit permission obtained 22 March 2018 - Link to permission
OSM attribution (if required): sortere.no
ODbL Compliance verified: Yes, explicit permission obtained
OSM data files
Update: The OSM file is now generated using this program: github.com/osmno/sortere2osm
A JSON datafile has been retrieved using data.sortere.no/api/docs/ and is available on this folder:
- sortere.json - the original json file
- sortere.osm - the file after transformation
- sortere.xlsx - the file in tabular format
Please note that some attributes will not be imported (ref. tagging table below), but has been included here for reference.
Most locations are very accurate. Some locations are off by up to 100 meters, a few even more. 29 locations do not have coordinates. The various partcipating recycling companies are not tagging the recycling categories 100% consistently, so there will be some variation regarding the level of detail in the recycling:xxx=yes tag.
Import type
The dataset will initially be imported by one person, then each county will be verified by members of the local community.
Data preparation
Data reduction and simplification
Each recycling type has been reduced into one recycling:xxx=yes tag. All recycling containers and centres are one node only each.
Tagging plans
All recycling centres and containers will have the following tagging:
| Original data | OSM tagging | Comment |
|---|---|---|
| kind=gjenvinningsstasjon | amenity=recycling +recycling_type=centre | Recycling centre |
| kind=returpunkt, hyttepunkt, batpunkt, farlig, pantepunkt | amenity=recycling +recycling_type=container | Local recycling container |
| overskrift | name=*description=* | Name: For recycling centres (not used for containers). The names will be reviewed and copy-edited before import.
Description: For containers (not used for centres). Content not copy-edited. |
| ikon=hyttepunkt | access=permissive | Reserved for vacation homes (may need key) |
| gateadresse postnr |
Address. Will not be uploaded to OSM. | |
| telefon | phone=* | Phone. For centres only. |
| apningstid | Opening hours. Will not be uploaded to OSM because the quality is questionable. | |
| id | ref:sortere=* | Sortere.no identification. Will be used to discover changes during later updates. |
| kind / ikon | Type of recycling site (will not be imported to OSM):
| |
| beskrivelse | Description of location or waste type.
This attribute contains HTML format coding. Will not be uploaded to OSM but provided in the dataset for information. | |
| kommune | Municipality. Will not be uploaded to OSM. | |
| kommunenr | Municipality id. Will be used to group nodes during import into county partitions. Will not be uploaded to OSM, but used to create a county attribute for help during import. | |
| Lat
Lng |
latitude
longitude |
WGS84 coordinates |
Translation table
Each node will also have recycling:xxx=yes tagging according to the following translation table:
| Original ref | Original data | OSM tagging | Number of sites | Usage in OSM |
|---|---|---|---|---|
| 2 | Annet / Other | (none) | 31 | |
| 9 | Batterier / Batteries | recycling:batteries | 483 | |
| 13 | Biler / Cars | recycling:cars | 8 | |
| 17 | Bygge- og rivningsavfall / Waste from construction and demolition | recycling:rubble | 360 | |
| 18 | Bølgepapp / Corrugated cardboard | recycling:paper | 660 | |
| 24 | Dekk / Tyres | recycling:tyres | 320 | |
| 26 | Drikkekartong / Beverage carton | recycling:paper | 854 | |
| 29 | Elektrisk og elektronisk avfall / Electrical items waste | recycling:electrical_items | 422 | |
| 31 | Emballasjekartong / Cardboard packaging inserts | recycling:paper | 759 | |
| 37 | Farlig avfall / Hazardous waste | recycling:hazardous_waste | 540 | |
| 40 | Flasker og bokser med pant / Bottles, cans with deposit-refund (usually donated to charity) Container deposit legislation
|
recycling:refund_bottles | 121 | |
| 44 | Gjenstander til ombruk / Items for second hand use | recycling:second_hand | 116 | |
| 46 | Glassemballasje / Glass packaging (jars etc) | recycling:glass_bottles | 4017 | |
| 48 | Grovavfall / Large items (too big for container) | recycling:rubble | 359 | |
| 50 | Hageavfall / Garden waste | recycling:garden_waste | 368 | |
| 56 | Impregnert trevirke / Impregnated wood | recycling:wood | 378 | |
| 58 | Jern og stål / Scrap metal | recycling:scrap_metal | 243 | |
| 75 | Matavfall / Food waste | recycling:organic | 615 | |
| 78 | Medisinsk avfall / Medical waste | recycling:drugs | 33 | |
| 80 | Metallemballasje / Light metal packaging (cans etc) | recycling:glass_bottles | 3948 | |
| 82 | Metall / Metal | recycling:metal | 410 | |
| 97 | Papir / Paper | recycling:paper | 985 | |
| 99 | Papp, papir og kartong / Cardboard and paper | recycling:paper | 1255 | |
| 104 | Plastemballasje / Plastic packaging | recycling:plastic_packaging | 992 | |
| 110 | Restavfall / Residual, general waste | recycling:waste | 2605 | |
| 113 | Sammensatt jern og stål / Combination of metals | recycling:metal | 220 | |
| 130 | Tekstiler og sko til ombruk / Clothes and shoes for second hand use | recycling:clothes | 1261 | |
| 131 | Trevirke / Wood | recycling:wood | 392 | |
| 137 | Vindusruter / Window glass | recycling:glass | 266 | |
| 143 | Eksplosjonsfarlig avfall / Explosive waste | recycling:hazardous_waste | 12 | |
| 167 | Deponirest / Waste which cannot be decomposed any further and which cannot be recycled | recycling:hazardous_waste | 53 | |
| 168 | Ikke brennbart avfall / Non-burnable waste | recycling:waste | 147 | |
| 169 | Isopor (EPS) / Polystyrene | recycling:polystyrene_foam | 260 | |
| 173 | Blandet plast / Mixed plastic waste | recycling:plastic | 187 | |
| 174 | Gips / Plasterboard | recycling:plasterboard | 178 | |
| 176 | Plast (ikke emballasje) / Plastic (not packaging) | recycling:plastic | 188 | |
| 177 | Småelektronikk / Small appliances | recycling:electrical_items | 214 | |
| 180 | Brennbart restavfall / Burnable residual waste | recycling:waste | 348 | |
| 184 | Rene masser / Clean, solid natural substances (dirt, rocks etc.) | recycling:rubble | 48 | |
| 185 | Inert avfall / Inert waste, i.e. with no physical, chemical or organic transformation (bricks, concrete, porcelain, glass etc) | recycling:rubble | 59 | |
| 186 | Kompostjord / Compost (for sale at the recycling centre) | (none) | 23 | |
| 191 | Asbest / Asbestos | recycling:asbestos | 158 | |
| 192 | Marint avfall / Marine debris, waste recovered along the shores | recycling:marine_waste | 94 | |
| 208 | Små fritidsbåter / Small leisure boats | recycling:boats | 165 | |
| 209 | Store fritidsbåter uten innenbordsmotor / Large leisure boats without inboard engine | recycling:boats | 21 | |
| 210 | Fritidsbåter med innenbordsmotor / Large leisure boats with inboard engine | recycling:boats | 18 |
Changeset Tags
The changesets will be uploaded using an import account and tagged with:
- description=Recycling import
- source=Sortere.no
- source:date=2018-03-22
Data transformation
Update: The OSM file is now generated using this program: github.com/osmno/sortere2osm
This is the workflow for the data transformation to create an OSM file from the json dataset:
- Download the dataset from data.sortere.no/api/punkter/husholdning
- Run
sed -f sorteretagging.txt sortere.json >sortere_simplified.json - Convert into CSV using convertcsv.com/json-to-csv
- Load csv file into Excel and save as Excel 97 xls file
- With the Opendata plugin enabled, load the xls file into JOSM
- Save as OSM file
Data transformation results
The processed OSM file is called sortere.osm and is stored in this folder.
Please note that some of the attributes, such as description=* and opening_hours=* will not be imported into OSM.
Data merge workflow
Team approach
The dataset will initially be imported by one person, then locations will be verified by the local community.
References
Recycling locations may be verified at the sortere.no website, including a more detailed description for each recycling type.
Workflow
Here is a typical workflow during import, including for later updates:
- Prepare a new file with recycling sites according to the Data transformation section above
- Store the file with the date of the retrieval in order to make diffs next time
- Using the Conflation plugin in JOSM:
- Select all sites from the file and define as Reference in Conflation
- In a different layer, download amenity=recycling using the Overpass API and set as Subject
- In the Configure window, set Distance to 100 meters, untick Replace Geometry and then generate matches
- Walk through the list of (initially approx. 500) matches and decide what to do:
- If the match is positive, hit the Conflate button. Please note that the existing site will not be relocated. You may move it manually if you notice that the location should be altered. Conflation will automatically keep the existing recycling:xxx=yes tags and add any new tags.
- If the match is negative, manually copy the new recycling site to the Subject layer and hit the Remove button
- When all matches have been processed, select all locations in the Reference only window and copy them to the Subject layer
- Upload to OSM
Conflation
Conflation is described in the Workflow section above.
QA
Recycling locations may be verified at the sortere.no website, including a more detailed description for each recycling type.
Locations in OSM are displayed on this map: Gjenvinning (Norwegian)
See also
- Municipal solid waste on Wikipedia
- Tag description: amenity=recycling
- sortere.no
- data.sortere.no/api/docs/ - Sortere.no API documentation
- Gjenvinning - Recycling containers and centres in OSM (Norwegian)
- Github: sortere2osm