Wikidata

From OpenStreetMap Wiki
Jump to navigation Jump to search
Wikidata-logo-en.svg

Wikidata is a free knowledge base driven by a community and hosted by the Wikimedia Foundation. As well as providing a repository for linked open data it aims to support Wikipedia and its sibling projects (such as Wikivoyage). Like Wikipedia, anyone can edit Wikidata. Also like Wikipedia, it is a collection of citations rather than an original source. Wikidata data is provided under a CC0 ("public domain") licence. Note that importing data from Wikidata into OSM is generally not feasible due to copyright incompatibilities (see that section for details).

Why link to Wikidata?

Sir Tim Berners-Lee devised the “Five stars of open data”, the fifth, and highest, level of which is “link your data to other data to provide context”. By including a Wikidata ID (which software understands as a URI) in an OSM object we link it to an object (called an "item") on Wikidata, and from there, directly and indirectly, to many other Wikidata items and to objects in many other systems. That makes OSM part of the Linked Open Data graph rather than an isolated system.

More pragmatically, it allows anyone querying OSM to retrieve additional data and links to other databases about an OSM object. That might, for example, be the population of a town, both current and at various periods in its history, or the birth date and place of the architect of a building, and their portrait, or a link to a system showing the last week’s rainfall in a nature reserve. In time, this may enable more complex queries like “all the scientific papers written by people born in this village, sorted by how many other papers cite them”. (Some of this third party data is under a licence compatible with OSM and some is not, and hence may not be as reusable. Caveat emptor.)

The ability to use Wikidata as an 'ID broker' should prevent the proliferation of ref tags for external systems/databases on OSM objects and the subsequent burden of maintenance. There will often be debate whether a specific system's refs are significant enough to be part of OSM or whether it is more appropriate to rely on Wikidata.

Including Wikidata tags in OSM means that OSM objects can be retrieved and used by other systems such as the highlighted infoboxes in Wikimedia Commons, etc. (see Usage section).

Wikidata is *not* Wikipedia

Wikidata contains data from many sources other than Wikipedia, and whilst many Wikidata items were originally generated from Wikipedia pages, Wikidata have re-labelled many of the dismabigution-named Wikipedia articles to be more like a database.

Wikidata items point to multiple language Wikipedia articles, which may be rather different. The power of Wikidata is the cited data that it itself contains and not just the linked content of Wikipedia.

Linking from OSM to Wikidata

WdOsm-semanticBridge.jpg

Wikidata entities can be linked to any kind of OSM object via the wikidata=* key. The key has a format 'Qnnnnn', e.g. Q936 which is OpenStreetMap. The objects are language independent, but the keys are not human-readable, unlike other things in OSM.

There is not a 1-to-1 mapping between OSM and Wikidata. A Wikidata item may refer to several different OSM database objects, for example: a (boundary=administrative) municipality, place=neighborhood, place=city, and place=island might have the same name but be mapped differently in OSM. Some Wikidata items represent features which are mapped as a series of several objects in OSM (e.g. long streets, streams)

Another way to link objects is by using wikipedia=* however Wikidata is much more atomic and database-like than Wikipedia. The wikipedia=* tag has the advantage of being human-readable.

Duplicate items may be created on Wikidata, discovered later, and merged. One of them (usually the younger) becomes a redirect. OSM should not point to a redirect object as this may lead to the link being ignored in some queries.

As a tag qualifier

Main article: Key:wikidata#Secondary Wikidata links

Tools

Tools like WIWOSM are supporting this way. With wikidata tag, data can be easily checked with the Wikidata+OSM SPARQL query service, which could show any discrepancies and errors.

wikidata=* item identifiers can be added to OSM elements by using:

  • The ID online editor – see “Connecting OpenStreetMap and Wikidata” (Mapbox blog post).
  • The JOSM editor with these additional plugins:
    • JOSM/Plugins/Wikipedia – has a button to query Wikidata and can open a selected Wikidata item in browser.
      Note that coordinates can be copied directly from within JOSM to a Wikidata item; just select an appropriate node, press Ctrl+ Shift+C then paste to the “Geographical coordinates” field of the item.
    • JOSM/Plugins/Tag2Link – open Wikidata item in browser.
  • OSM Wikidata Explorer – visualize distance between OSM and Wikidata coordinates (data doesn't appear to have been updated since summer of 2017).
  • OSMgadget – a Mediawiki gadget to pull in OSM data while viewing Wikipedia.
  • osm.wikidata.link – detects possible matches and allows quickly adding them to OSM.
  • Wikipedia where in OSM is a project to show for a Wikipedia article geometric objects from OpenStreetMap.

Validation tools:

Note that in certain cases, wikidata=* should be used as a suffix of another key. For example, if you want to link an OSM shop to the Wikidata object of the brand, you can use brand:wikidata=*; a statue can be tagged artist:wikidata=* and subject:wikidata=*, etc.

Wikidata user scripts

Several user scripts on Wikidata allow to query and render an OSM-derived map for specific items in Wikidata matching OSM elements:

  • Overpass” – adds to a Wikidata item page a map showing features tagged in OSM with their Wikidata item identifier, with a link to view the object using Overpass Turbo
  • osm” shows a link in the sidebar of a Wikidata item, if its QiD is used in a tag on OSM
  • MapView” – displays an OSM-derived map on Wikidata item page with coordinates.

Linking from Wikidata to OSM

  • OSM Relation IDs can be added to Wikidata records by using Property P402
  • Ways and nodes are not considered stable IDs by Wikidata because, for example, a node may be deleted and re-added as a polygon. (In practise does this happen significantly often?)
  • A Permanent ID was proposed to solve this issue.
  • Linked OSM relations can be listed with: Special:WhatLinksHere/Property:P402 or here as query.
  • Wikidata contains 105552 OSM IDs as of 31 Jan 2020.

OSM tags and keys

For a list of OSM tags’ and keys’ equivalent items in Wikidata, see what links to Property:P1282.

Usage by web sites, etc.

Wikimedia Foundation

Wikimedia Foundation projects, including each language’s Wikipedia, display maps using OSM data. If an entity in OSM has been tagged with a Wikidata ID, its outline can be highlighted on the map. At the moment, there is no POI support. Only shapes and lines can be shown this way. See w:mw:Help:Extension:Kartographer#External_data.

OSM areas (not points or lines) with wikidata tags are highlighted in infoboxes in Commons etc.

The Kartographer extension binds OSM objects to their Wikipedia article when the object has a wikidata=* tag, see for example this article map in the infobox.

If the Wikidata ID tagged on an OSM object is to a redirect then the object will not be highlighted on Wikimedia maps.

UMap

Reasonator

Reasonator, which puts a human-friendly front end on Wikidata, is now showing two links for geotagged entries, to TagInfo and Overpass turbo, querying these tags.

For example, the Reasonator page:

http://tools.wmflabs.org/reasonator/?q=Q915614

has links to:

https://taginfo.openstreetmap.org/tags/wikidata=Q915614
http://overpass-turbo.eu/?w=%22wikidata%22%3F%22Q915614%22+global&R

You’ll see them just below the map images.

Kartographer

OSM elements tagged with Wikidata item unique identifiers can be displayed in Wikipedia pages, as overlays on OSM-based maps, using the Kartographer Extension. See also Kartographer/OSM.

OSM wiki

Templates to create links to Wikidata pages:

  • {{Q}} – display a link to a Wikidata item by its unique (hopefully stable) identifier.
  • {{Property}} – display a link to a Wikidata property by its unique (hopefully stable) identifier.

Querying Wikidata and OSM

Wikidata data can be retrieved in a number of ways, one of which is SPARQL. Since OSM data can also be queried using SPARQL, Wikidata “federates” the two, allowing them to be queried jointly.

Specific OSM-based services which Wikidata federates with include:

Importing data

Importing data from OSM into Wikidata

Copying data from OSM to Wikidata (or even from other Wikimedia projects) is not allowed because Wikidata uses the public-domain style Creative Commons CC0 license which does not contain any attribution or share-alike provisions. Obviously, this does not apply to original data which you have generated yourself, for example by your own surveys.

Note Wikimedia legal team advice confirms this: For EU databases, bots or other automated ways of extracting data should also be avoided because of the Directive’s prohibition on “repeated and systematic extraction” of even insubstantial amounts of data. (from w:m:Wikilegal/Database Rights#Conclusion).

Importing data from Wikidata into OSM

Wikidata is using the CC0 license, which on first sight allows imports into OSM. However, OSM does not, generally, accept imports of data from Wikidata due to the sources often used to contribute to Wikidata. A significant part of Wikidata data was harvested from Wikipedia, in which for example most coordinates are produced using/copied from products with licenses conflicting with OSM.

Note that CC0 license "only extends to material the licensor actually has rights in and specifically avoids making a statement on the status of any third party material included. For use in OpenStreetMap the licences of all the data that is included in the dataset need to be determined and compatibility verified."[1]

For example copying from Google Maps to OpenStreetMap (even indirectly) is not OK, and copying Wikidata coordinates (that were copied to Wikipedia from Google Maps) is also not OK. Note that Wikipedia allows and encourages to use Google Maps to obtain coordinate data[2] and other sources that are not OK to be used in OSM[3].

The same applies for any other data present in Wikidata.

Import are OK only for Wikidata content that was imported from sources compatible with OSM - what excludes for example coordinates obtained from Wikipedia.

OpenStreetMap tries to be whiter than white, and data should be legally usable in any country[4]. Therefore OpenStreetMap acknowledges sui generis database right. As a conclusion, an import of information from Wikidata into OSM is generally not permitted because Wikidata has lower standards in terms of copyright than OSM. In addition, Wikidata contains some data from sources we are not allowed to use[5].

In rare situations, when it is possible to verify that data has acceptable copyright status it is possible to use part of Wikidata data.

Past activities

  • 2013-09-12: There was a proposal to automatically add Wikidata IDs to a number of OSM objects.
  • 2016-10-27: Mapbox has proposed a procedure for importing, but has yet to get community feedback on it and thus reverted in the meantime.
  • 2016-11-17: There has been a Mapbox inspired mass-edit but has also been put on hold until further community discussion.

Academic papers about OSM

Wikidata has details of academic papers about OSM. They can be seen at Scholia.

Notes and references

See also