Spanish Cadastre/Buildings Import/Data Conversion/Software
|Proposal||Import Guide||Corrections||Projects management||Software|
The installation procedure is described here.
The software by default use Spanish to translate the throughfare types. To use another language, edit the file 'setup.py'. Change 'es' to 'cat' for Catalan, or to 'gl' for Galician, in this lines:
# Dictionary for default 'highway_types.csv' highway_types = highway_types_es
# List of highway types to translate as place addresses place_types = place_types_es
# List of place types to remove from the name remove_place_from_name = [place_types_es]
Basic use (buildings)
The program is executed in the command shell, from the folder that you want to dedicate to downloading the data.
To download a municipality you need its Cadastral code. If you don't know it, you can run the program with the -l option and the code of a province (two digits) to list all municipalities in the province.
catatom2osm -l 02
Territorial Office 02 - Albacete ================================================================== 02001-ABENGIBLE 02002-ALATOZ 02900-ALBACETE ... 02084-VILLAVERDE DE GUADALIMAR 02085-NURSERIES 02086-YESTE
Once you know the code, you can try the simplest option:
catatom2osm -tm 02069
The program will create the folder '02069', download there the necessary zip files from Cadastre and generate some files. You can review the file 'report.txt' and take a look at the OSM files in the 'tasks' folder.
Addresses conversion and conflation
If you use the program with the default options, in addition to the buildings it will try to convert the addresses data and conflate it with the OSM data:
The program ends indicating that a file for converting the thoroughfare names ' highway_names.csv' has been generated and you must check it as part of the process of Review of thoroughfare names (es).
After you have checked the conversion file, you can run the program again to continue the process. As a result, the OSM files in 'tasks' will now contain the addresses in addition to the buildings.
Access to facade photos
The -d option convert the addresses into a separate file 'address.osm' apart from the buildings. Don't import it, but you can use it as a source to obtain missing addresses in OSM and place them in the appropriate place manually.
catatom2osm -d 02069
This file contains the links to the front photos of each parcel that are used to check the house numbers and additional information about the buildings. That is why it is also generated with the -b and -t options. In order to visualize the photos it is necessary to have enabled the Tag2Link plugin. With the file opened in JOSM, select a node and in the Selection List Dialog right click on the entry for this node in the list. Select the View image in the contextual menu to open the image in your browser.
The image = * tag should not be uploaded to OSM.
This is not the only method to access the images, the complement of Josm pointinfo allows you to do it more comfortably.
-b: A single 'building.osm' file is generated with all the buildings. Don't import it, but you can use it to inspect/validate the results for the municipality as a whole. For the largest municipalities, take into account that this option consumes more memory than -t.
-z: To generate only the boundaries of polygons and blocks for the task manager. Don't import it, but it could serve as a guide to draw barriers and areas using an aerial background image.
-p: A file 'parcel.osm' is generated with the cadastral parcels. Don't import it.
Additional information on the transformation of thoroughfare names
When the program does not locate the name of a thoroughfare in OSM, it uses the one from Catastro after transforming it. Here some details of how this transformation is performed are explained, but they are not too important since the idea is that in OSM the correct names are previously introduced and the program will take them from there.
The values for the translation of the road types are read from the 'highway_types.csv' file that you can optionally complete or adapt.
Other aspects of the translation of names that can be configured in the 'setup.py' file are the following:
The 'report.txt' file contains the following sections:
Contains data obtained from the administrative boundary relation of the municipality. The program tries to locate this relation searching in OSM for the municipality name closest to the name in Cadastre. It is important because the administrative boundary is used to limit the data downloaded from OSM for conflation. In most cases the matching is correct, but both names are showed to confirm that there is no error. If the name obtained from OSM is not correct, see error when locating the municipality. If 'None' appears as the name of the municipality see The administrative limit could not be obtained, the bounding box will be used. When available, the population data and the Wikipedia/Wikidata links appear. We can take this opportunity to confirm and/or update them.
Municipality: La Roda Cadastre name: LA RODA Code: 02069 Surface: 397.8 km² Population: 16299 hab. (2009) Wikipedia: https://www.wikipedia.org/wiki/es:La Roda (Albacete) Wikidata: https://www.wikidata.org/wiki/Q630706 Date: 15/11/17
Contains data about the versions of the program, the operating system, the memory and processor of the system, time and resources used.
Application version: CatAtom2Osm 2017-11-14 Platform: Linux 4.4.0-96-generic #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017 x86_64 x86_64 QGIS version: 2.14.11-Essen CPU count: 4 CPU frequency: 3600.0 Mhz Execution time: 153.54 seconds Total memory: 7928.30 GB Physical memory usage: 251.81 GB Virtual memory usage: 1014.31 GB
Contains information about the addresses with these sections:
Date of publication and number of features in the Cadastral data set.
Source date: 2017-09-05 Feature count: 6016 Type entrance: 4710 Type parcel: 1306 Postal codes: 2 Street names: 237
Count of features deleted/transformed by the program.
Addresses without house number deleted: 547 Addresses without associated building deleted: 537 Addresses belonging to multiple buildings deleted: 1165 'Parcel' addresses not unique for it building deleted: 2
Objects existing in OSM and conflicts.
OSM addresses: 882 Addresses rejected because they exist in OSM: 471
Elements generated in the output files.
Addresses: 3294 In entrance nodes: 2316 In buildings: 978 Type addr:street: 3290 Type addr:place: 4
Source date: 2017-09-05 Feature count: 22685 Buildings: 6103 Buildings parts: 16265 Swimming pools: 317
Parts outside footprint deleted: 512 Parts with no floors above ground: 265 Building footprints created: 5 Buildings with multipart geometries: 1689 Buildings resulting from splitting multiparts: 4065 Parts merged to the footprint: 10000 Adjacent parts merged: 497 Spike vertices deleted: 4 Close vertices merged: 324 Topological points created: 7494 Simplified vertices: 18122
Buildings/pools in OSM: 748 With conflict: 606
Nodes: 72292 Ways: 14247 Relations: 451 Feature count: 13792 Buildings: 8484 Buildings parts: 4991 Swimming pools: 317 Building types counter: industrial: 814, office: 10, residential: 4216, yes: 2434, retail: 78, ruins: 64, public: 90, barn: 778 Max. levels above ground (level: # of buildings): 1: 2629, 2: 2634, 3: 492, 4: 135, 5: 94, 6: 46, 7: 16, 8: 1, 9: 1, 10: 1, 101: 1 Min. levels below ground (level: # of buildings): 1: 397, 2: 12, 3: 1 Rustic tasks files: 114 Urban tasks files: 326
This is the most important part in the report, it details issues that might require our action.
Number of fixmes reported in the OSM files. You must review them and delete the 'fixme' tag before uploading the file. You can find:
The program performs some checks adding the values obtained in the report. If the results do not match, the problem is reported. This does not usually happen, notify the developer.
The checks carried out are:
When using the program you may find the following problems.
Manually download data for conflation
Sometimes, particularly in large municipalities, the Overpass servers may be overloaded. If the program can't download the data for conflation you can try using the --log=DEBUG option. This way, the url of the Overpass query is exposed. Copy and paste it in your browser and save the result with the names 'current_highway.osm', 'current_address.osm' or 'current_building.osm' according to the case.
Bad match of municipality name
The name in the 'Municipality' line 'report.txt' doesn't match with the correct municipality. This mean that the program has failed to find the name with greater similarity and proximity in OSM to the name in Cadastre. To correct it, locate in OSM the identifier of the administrative boundary relation of the municipality and set it in the 'mun_fails' option at the end of the 'setup.py' file. The search algorithm has been tested for all the municipalities, so this situation should be exceptional.