Import/Catalogue/Recycling Import Norway

From OpenStreetMap Wiki
Jump to: navigation, search

Goals

The goals of this import are to:

  • Add all recycle centers and containers from Sortere.no to OSM, providing approx. 6.000 new recycling locations in addition to the approx. 1.000 recycling locations already in OSM.
  • Add recycling types to each location

Schedule

  • 22 March 2018 - Explicit permission and data obtained from Loop/Sortere.no.
  • 3 April 2018 - Import plan discussed at the Norwegian mailing list - Link
  • 6 April 2018 - Import plan submitted to the Import mailing list.
  • 15 April 2018 - Dataset uploaded to OSM - Map on Gjenvinning (Norwegian)

Import data

Background

The various regional recycling companies in Norway have establish Loop, a foundation for promoting recycling. The Norwegian Directorate for Environment Protection is co-funding the activities. The foundation has established a joint database for all recycling sites which is being updated regularly by its member companies. The database is presented on the web site sortere.no and it is available to app developers and map services. OSM is the first organisation to receive the opportunity to use the dataset as open data without any written agreement.

Data source site: Dataset provided from data.sortere.no/api
Data license: No license defined
Type of license (if applicable): None
Link to permission (if required): Explicit permission obtained 22 March 2018 - Link to permission
OSM attribution (if required): sortere.no
ODbL Compliance verified: Yes, explicit permission obtained

OSM data files

A JSON datafile has been retrieved from data.sortere.no/api and is available on this folder:

  • sortere.json - the original json file
  • sortere.osm - the file after transformation
  • sortere.xlsx - the file in tabular format

Please note that some attributes will not be imported (ref. tagging table below), but has been included here for reference.

Most locations are very accurate. Some locations are off by up to 100 meters, a few even more. 29 locations do not have coordinates. The various partcipating recycling companies are not tagging the recycling categories 100% consistently, so there will be some variation regarding the level of detail in the recycling:xxx=yes tag.

Import type

The dataset will initially be imported by one person, then each county will be verified by members of the local community.

Data preparation

Data reduction and simplification

Each recycling type has been reduced into one recycling:xxx=yes tag. All recycling containers and centres are one node only each.

Tagging plans

All recycling centres and containers will have the following tagging:

Original data OSM tagging Comment
type=gjenvinningsstasjon amenity=recycling +recycling_type=centre Recycling centre
type=returpunkt, hyttepunkt, batpunkt, farlig, pantepunkt amenity=recycling +recycling_type=container Local recycling container
overskrift name=*description=* Name: For recycling centres (not used for containers). The names will be reviewed and copy-edited before import.

Description: For containers (not used for centres). Content not copy-edited.

type=hyttepunkt access=private Reserved for vacation homes (may need key)
gateadresse

postnr
poststed

addr:street=*

+addr:housenumber=* +addr:postcode=* +addr:city=*

Address. Will not be uploaded to OSM.
telefon phone=* Phone. Will not be uploaded to OSM.
apningstid opening_hours=* Opening hours. Will not be uploaded to OSM because the quality is questionable.
id sortere:ref=* Sortere.no identification. Will be used to discover changes during later updates.
type sortere:type=* Type of recycling site (will not be imported to OSM):
  • returpunkt (regular container)
  • hyttepunkt (container for vacation homes)
  • pantepunkt (container for deposit/refund bottles)
  • farlig (container for hazardous waste)
  • batpunkt (recycling centre for boats)
  • gjenvinningsstasjon (recycling centre)
beskrivelse sortere:beskrivelse=* Description of location or waste type.

This attribute contains HTML format coding. Will not be uploaded to OSM but provided in the dataset for information.

kommune sortere:kommune=* Municipality. Will not be uploaded to OSM.
kommunenr sortere:kommunenr=* Municipality id. Will be used to group nodes during import into county partitions. Will not be uploaded to OSM.
Lat

Lng

latitude

longitude

WGS84 coordinates

Translation table

Each node will also have recycling:xxx=yes tagging according to the following translation table:

Original ref Original data OSM tagging Number of sites Usage in OSM
2 Annet / Other (none) 31
9 Batterier / Batteries recycling:batteries 483
13 Biler / Cars recycling:cars 8
17 Bygge- og rivningsavfall / Waste from construction and demolition recycling:rubble 360
18 Bølgepapp / Corrugated cardboard recycling:paper 660
24 Dekk / Tyres recycling:tyres 320
26 Drikkekartong / Beverage carton recycling:paper 854
29 Elektrisk og elektronisk avfall / Electrical items waste recycling:electrical_items 422
31 Emballasjekartong / Cardboard packaging inserts recycling:paper 759
37 Farlig avfall / Hazardous waste recycling:hazardous_waste 540
40 Flasker og bokser med pant / Bottles, cans with deposit-refund (usually donated to charity)
Container deposit legislation
recycling:refund_bottles 121
44 Gjenstander til ombruk / Items for second hand use recycling:second_hand 116
46 Glassemballasje / Glass packaging (jars etc) recycling:glass_bottles 4017
48 Grovavfall / Large items (too big for container) recycling:rubble 359
50 Hageavfall / Garden waste recycling:garden_waste 368
56 Impregnert trevirke / Impregnated wood recycling:wood 378
58 Jern og stål / Scrap metal recycling:scrap_metal 243
75 Matavfall / Food waste recycling:organic 615
78 Medisinsk avfall / Medical waste recycling:drugs 33
80 Metallemballasje / Light metal packaging (cans etc) recycling:glass_bottles 3948
82 Metall / Metal recycling:metal 410
97 Papir / Paper recycling:paper 985
99 Papp, papir og kartong / Cardboard and paper recycling:paper 1255
104 Plastemballasje / Plastic packaging recycling:plastic_packaging 992
110 Restavfall / Residual, general waste recycling:waste 2605
113 Sammensatt jern og stål / Combination of metals recycling:metal 220
130 Tekstiler og sko til ombruk / Clothes and shoes for second hand use recycling:clothes 1261
131 Trevirke / Wood recycling:wood 392
137 Vindusruter / Window glass recycling:glass 266
143 Eksplosjonsfarlig avfall / Explosive waste recycling:hazardous_waste 12
167 Deponirest / Waste which cannot be decomposed any further and which cannot be recycled recycling:hazardous_waste 53
168 Ikke brennbart avfall / Non-burnable waste recycling:waste 147
169 Isopor (EPS) / Polystyrene recycling:polystyrene_foam 260
173 Blandet plast / Mixed plastic waste recycling:plastic 187
174 Gips / Plasterboard recycling:plasterboard 178
176 Plast (ikke emballasje) / Plastic (not packaging) recycling:plastic 188
177 Småelektronikk / Small appliances recycling:electrical_items 214
180 Brennbart restavfall / Burnable residual waste recycling:waste 348
184 Rene masser / Clean, solid natural substances (dirt, rocks etc.) recycling:rubble 48
185 Inert avfall / Inert waste, i.e. with no physical, chemical or organic transformation (bricks, concrete, porcelain, glass etc) recycling:rubble 59
186 Kompostjord / Compost (for sale at the recycling centre) (none) 23
191 Asbest / Asbestos recycling:asbestos 158
192 Marint avfall / Marine debris, waste recovered along the shores recycling:marine_debris 94
208 Små fritidsbåter / Small leisure boats recycling:boats 165
209 Store fritidsbåter uten innenbordsmotor / Large leisure boats without inboard engine recycling:boats 21
210 Fritidsbåter med innenbordsmotor / Large leisure boats with inboard engine recycling:boats 18

Changeset Tags

The changesets will be uploaded using an import account and tagged with:

description=Recycling import
source=Sortere.no
source:date=2018-03-22

Data transformation

This is the workflow for the data transformation to create an OSM file from the json dataset:

  1. Download the dataset from data.sortere.no/api/punkter/husholdning
  2. Run sed -f sorteretagging.txt sortere.json >sortere_simplified.json
  3. Convert into CSV using convertcsv.com/json-to-csv
  4. Load csv file into Excel and save as Excel 97 xls file
  5. With the Opendata plugin enabled, load the xls file into JOSM
  6. Save as OSM file

Data transformation results

The processed OSM file is called sortere.osm and is stored in this folder.

Please note that some of the attributes, such as description=* and opening_hours=* will not be imported into OSM.

Data merge workflow

Team approach

The dataset will initially be imported by one person, then locations will be verified by the local community.

References

Recycling locations may be verified at the sortere.no website, including a more detailed description for each recycling type.

Workflow

Here is a typical workflow during import, including for later updates:

  1. Prepare a new file with recycling sites according to the Data transformation section above
  2. Store the file with the date of the retrieval in order to make diffs next time
  3. Using the Conflation plugin in JOSM:
    • Select all sites from the file and define as Reference in Conflation
    • In a different layer, download amenity=recycling using the Overpass API and set as Subject
    • In the Configure window, set Distance to 100 meters, untick Replace Geometry and then generate matches
    • Walk through the list of (initially approx. 500) matches and decide what to do:
    • If the match is positive, hit the Conflate button. Please note that the existing site will not be relocated. You may move it manually if you notice that the location should be altered. Conflation will automatically keep the existing recycling:xxx=yes tags and add any new tags.
    • If the match is negative, manually copy the new recycling site to the Subject layer and hit the Remove button
    • When all matches have been processed, select all locations in the Reference only window and copy them to the Subject layer
  4. Upload to OSM

Conflation

Conflation is described in the Workflow section above.

QA

Recycling locations may be verified at the sortere.no website, including a more detailed description for each recycling type.

Locations in OSM are displayed on this map: Gjenvinning (Norwegian)

See also