Import/Catalogue/sous-surveillance.net

From OpenStreetMap Wiki
Jump to navigation Jump to search

--- DRAFT VERSION OF AN IMPORT AT THE FEASABILITY STUDY STAGE ---

Cameras from surveillance.net is an import of a dataset build and managed by the website sous-surveillance.net. The dataset is of type GeoJson containing a FeatureCollection of cameras. The features are already, nearly compliant with man_made=surveillance. The dataset is covering many cities in France, Belgium, Luxembourg and also with less importance few cities elsewhere in the world (Montréal, Seattle, Moscow and Minsk). The import is currently (as of October 2019) at the "feasibility study" stage.

Context

Sous-surveillance.net was created 7 years ago in order to develop the inventorisation of cameras by local groups. For technical reasons, their inventory was not stored in Openstreetmap but in their own database. Their database was filled by multiple contributors and there is also a validation process for each camera. The size of their inventory is quite impressive as it includes around 20 000 cameras mostly located in France and Belgium where OSM includes around 80 000 cameras in the whole world. Today, they are ready to join the mapping effort with OSM. Therefore, discussions are currently happening with them to realize this import. Adaptation of their website to work with OSM is also discussed which would avoid future divergence of the databases and would bring new valuable indirect contributors to OSM. Note the final quantity of cameras imported in OSM will probably determined if sous-surveillance.net switches to OSM as back-end database. This should not push us to import their data at any price but we should consider carefully their valuable data.

Goals

The goal of the import is to merge the cameras inventoried by Sous-surveillance.net that are not already in Openstreetmap.

Schedule

✔️ 10/2019 - Discussion with sous-surveillance.net and Openstreetmap Belgium.
✔️ Proposition to sous-surveillance.net

  • License ready
  • Larger consultation to the Openstreetmap Community (email to the import mailing list and communication towards OSM France)
  • Available OSM upload file for review
  • Import in OSM

Import Data

Background

Data source site: https://www.sous-surveillance.net/
Data license: ??? The website will define a license ???
Type of license (if applicable): e.g. CC-BY-SA, Public Domain, Public Domain with Attribution, etc.
Link to permission (if required): e.g. link to mail list reference url - http://lists.openstreetmap.org/pipermail/imports/2012-December/001617.html
OSM attribution (if required): http://wiki.openstreetmap.org/wiki/Contributors#yourdataprovider
ODbL Compliance verified: yes/no

OSM Data Files

The OSM data file are available [???here???].

Import Type

The import will be a one-time import that will be prepared with QGIS and python and then performed in JOSM.

In python, the dataset will be converted according to the translation table. The ipynb is available [?? here ??]. In Qgis, the distance matrix between the sous-surveillance cameras and the OSM cameras will be calculated using vector analysis

Data Preparation

The source dataset is nearly compliant with the values associated to the man_made and the related tags. The data preparation is described in the translation table.

Tagging Plans

The cameras will be added as nodes according the following translation table:

Translation table

Original data OSM tagging Comment
surveillance:type=camera Will be added to every nodes
source=sous-surveillance.net Will be added to every nodes
survey:date=* Will be added based on the submission date of the camera on the sous-surveillance.net website.
apparence camera:type=* Type "dome" will be kept. "radar" will be not kept but surveillance:type will be set to "ALPR".

"boîte", "nue", "encastre" will be converted to "fixed" except if camera:feature=motion is set. Then, type will be set to camera:type=panning

camera:mount=* If apparence is set to "encastre". The value "wall" will be set.
direction camera:direction=*
angle camera:angle=*
camera_zoom camera:feature=* If camera_zoom is true, "zoom" will be added to the key camera:feature
camera_rotation camera:feature=* If camera_rotation is true, "motion" will be added to the key camera:feature
op_type surveillance=* "private" value will be converted to "outdoor" value
surveillance_zone surveillance:zone=* Values will be converted in one of the surveillance:zone value options (This will be only a translation)
title name=*
op_name operator=*
id_camera ref:sous-surveillance.net=*
lon longitude=* WGS84 coordinate
lat latitude=* WGS84 coordinate
description description=*
image image=*
zone Not imported

Note that some of the keys come from this proposal for the extension of man_made=surveillance : https://wiki.openstreetmap.org/wiki/Proposed_features/Extended_tags_for_Key:Surveillance

Changeset Tags

Data Merge Workflow

Team Approach

The import will be done by User:Vucodil via a dedicated account (User:sous_surveillance_import) with support of Openstreetmap Belgium.

Workflow

In the following, an osm camera is : man_made=surveillance and surveillance:type=camera. Several steps will be done to ensure a successful merge process :

  • Any camera from source that is closer than 5 meters from an OSM camera will excluded unless there is a closer camera in source.
  • Any camera from source that is located between 5 and 10 meters from an OSM camera will be marked with fixme=* = "may be a duplicated camera", unless there is a closer camera in source.
  • Any camera with a survey:date=* older than 10 years will be marked with fixme=* = "may be disused"

The other cameras will be directly added to OSM. In case of problem, the one-time import changeset will be reverted.

Following our current estimations:

  • 92.5% of the cameras will be directly imported (~16k)
  • 7% of the cameras will not be imported (~1k)
  • 0.5% of the cameras will be marked with fixme=* (~100)

Conflation

Conflation process is described in the workflow

QA

Reviewing process can performed in sous-surveillance.net if the website has migrated to Openstreetmap or on [1] where cameras are shown in details and where there is specific icons if the nodes contain the key fixme=*

See also