User talk:Panarchos

phyghtmap and OSM v0.6 format

2011-09-17 -- Warum schreibt phyghtmap die OSM Daten per default im v0.5 Format? Es gibt eigentlich kein Tool mehr, was das einlesen kann, so das man immer mühselig konvertieren muß. Kann phyghtmap nicht gleich das v0.6 Format schreiben? Ausser der Versionsnummer müssen bei den Nodes und Ways nur noch die Attribute "version" sowie "timestamp" hinzugefügt werden. Ich würde ja gerne selber einen Patch machen, aber durch den Python Code steige ich noch nicht so ganz durch. Ansonsten gefällt es mir bisher sehr viel besser als Srtm2Osm, vor allem wegen den sehr viel geringeren Speicherbedarf. [kukuk]

Ich habe phyghtmap v0.6 beigebracht (bekommt man mit --osm-version=0.6). 0.5 würde ich gerne deswegen als default behalten, weil es Plattenplatz spart und zum Rechnen von Garmin-Images mit mkgmap ausreicht. Ich mache ansonsten nicht viel mit den Höhenlinien -- wozu verwendet Ihr sie noch?

Soweit ich das sehe, ist das "timestamp"-Attribut nicht zwingend, nur das "version"-Attribut ist ein Design-Feature von Version 0.6 der OSM-API. Bitte korrigiert mich, wenn ich das falsch sehe. --Panarchos 12:58, 12 December 2011 (UTC)

phyghtmap Windows installation

Hi I just went trough your windows insallation. To have it working one has to install setuptools separately. Also for the other downloads I would recommend you link to the following page, that has all windows python tools: [1] Also you have to omit "python" in the setup command. Running "setup.py install" after chdir in an elevated (run as admin) cmd.exe is enough. Have not tried using it yet under windows though. It it ain't work I don't mind and dualboot to Ubuntu... just wanted to give it a try.--Extremecarver 12:46, 13 December 2011 (UTC)

I added your recommendations to the installation instructions. I would appreciate any feedback on how it works under windows.--Panarchos 19:10, 27 December 2011 (UTC)

Besides the above mentioned it works perfectly I think. Just not sure anymore, if there were no other special packages I needed to install. In case I find something else, I think I will simply directly edit it to the description if you don't mind. But probabely I won't - cause my installation already works - and I don#t want to reinstall just to find out.--Extremecarver 16:31, 28 December 2011 (UTC)

maxnodes configurable & pbf support

Im Prinzip siehe Titel. Es wäre super wenn phyghtmap größere Tiles schreiben könnte (mir sind sie derzeit vor allem mit viefinders 1" deutlich zu klein - sprich zu viele. Dazu wäre es auch super wenn phyghtmap pbf als Ausgabeformat unterstützen würde - weil a) es spart deutlich Zeit bei der Weiterverarbeitung b) es ist viel schneller und besser komprimierbar c) es spart sogar unkomprimiert deutlich Platz. --Extremecarver 12:23, 4 January 2012 (UTC)

Den Gedanken aufgreifend würd' ich vorschlagen, einen Parameter "--single-file" einzuführen, durch welchen in zwei temporäre Dateien Nodes und Ways getrennt in richtiger ID-Reihenfolge abgelegt würden und am Schluß zur einer Datei konkateniert (Einsparung der "Konvertierungs-skripte" wie etwa process_phyghtmap_data.pl!). Dieses Resultat hätte dann das "richtige OSM-Format" für osmconvert, welches sehr schnell sowohl Polygone ausschneidet wie auch pbf erzeugen kann (pbf ist bei diesen "OSM-Höhendaten" noch platzsparender als gzipt!). --Kellerma 08:57, 8 January 2012 (UTC)

In Version 1.30 gibt es die --max-nodes-per-tile Option. Die tut genau, was man denkt. Für eine Ausgabe in einer einzigen Datei kann man --max-nodes-per-tile=0 sagen. Außerdem ist das ausgegebene XML jetzt richtig sortiert: zuerst ein Block mit allen Knoten, dann ein Block mit allen Wegen. Alle IDs sind pro phyghtmap-Aufruf einzigartig und fortlaufend.--Panarchos 19:01, 22 January 2012 (UTC)

bounding polygon support

Im Prinzip braucht man osm v6 ja nur um Die daten mit osmconvert/osmosis mit einem Bounding Polygon noch einmal genauer zu schneiden. Wenn phyghtmap schon bounding polygons anstelle einer bounding box verarbeiten würde, wäre das nicht mehr nötig. --Extremecarver 12:23, 4 January 2012 (UTC)

osmconvert arbeitet sehr schnell _wenn_ die nodes & ways richtig sortiert im osm-file vorliegen. Es würde für phyghtmap daher genügen, das Ergebnis als "singe-file" (siehe oben) abzulegen. --Kellerma 09:03, 8 January 2012 (UTC)

Patches by Kukuk

Wouldn't it be good to integrate these patches/diffs [2] - especially the -max-nodes patch is essential...--Extremecarver 12:31, 4 January 2012 (UTC)

Done in version 1.30. There is a --max-nodes-per-way option.--Panarchos 19:05, 22 January 2012 (UTC)

Phyghtmap under Opensuse 11.4 (minimal)

Note that phyghtmap also depends on python-xml. So additionally to what is listed on the dependancies you have to add zypper install python-xml everything else runs smooth (and much quicker to setup than under windows). I think python-xml should also be added to the dependancies list for linux. --Extremecarver 15:55, 4 January 2012 (UTC)

I don't think so because I only use simple string operations to create the XML output. I don't use the xml module.--Panarchos 19:07, 22 January 2012 (UTC)

Phyghtmap >= 1.24

Phyghtmap => 1.24 does only work with pyhton matplotlib < 1.0.0. With matplotlib >= 1.0.0, you will always get: "TypeError: 'points' is an invalid keyword argument for this function". https://build.opensuse.org/package/files?package=phyghtmap&project=home%3Akukuk contains a patch "matplotlib-1.0.0.diff" which fixes this problem. The output is identical to phyghtmap 1.23 --Kukuk 15:43, 10 January 2012

Integrated in phyghtmap version 1.30.--Panarchos 19:08, 22 January 2012 (UTC)

Erstellung einer Höhenlinienkarte für Gesamteuropa

2012-02-02
Ich versuche eine Karte für Europa zu erstellen. Der Area-Parameter ist folgender: -a -11:34:32:72
Dabei gibt es aber ein Problem. Da ich gerne für Bereiche, wo es genauere Daten gibt, diese auch haben möchte, benutze ich den Schalter --viewfinder-mask=1. Das hat aber zur Folge, dass ich zwar für den Bereich südlich des 60. Breitengrades das erhalte, was ich will. Nördlich davon erhalte ich aber fast nichts, da es ja von der NASA dafür keine Daten gibt und von ViewFinderPanoramas nur die Daten mit guter Auflösung geladen werden, und davon gibt es im Norden meines Wissens nur den Bereich am Nordkapp.
Dann habe ich versucht das Ganze in zwei Schritten mit folgenden Aufrufen zu erstellen:

c:\python27\scripts\phyghtmap.exe -a -11:34:32:61 -s 10 -c 100,20 --start-node-id=2000000000 --start-way-id=1000000000 --corrx=0.0005 --corry=0.0005 --max-nodes-per-tile=2000000 --viewfinder-mask=1 --gzip=7

c:\python27\scripts\phyghtmap.exe -a -11:61:32:72 -s 10 -c 100,20 --start-node-id=4000000000 --start-way-id=1100000000 --corrx=0.0005 --corry=0.0005 --max-nodes-per-tile=2000000 --viewfinder-mask=3 --gzip=7

Jetzt habe ich zwar Daten für das komplette Gebiet, allerdings wieder mit zwei Einschränkungen: Nördlich des 60. Breitengrades bekomme ich nur eine Auflösung von 3 Grad, hätte es aber so genau, wie Daten verfügbar sind. Das zweite Problem ist schwerwiegender, beim Erstellen der endgültigen Karte mit mkgmap, ergibt sich eine Lücken zwischen den beiden Bereichen (am 60. Breitengrad) von ca. 70 km, wo es keine Höhenlinien gibt (warum auch immer).
Gut wäre es, wenn man den Bereich in einem Schritt erstellen könnte.

Frage und Vorschlag:
Ist es möglich das Programm so zu erweiteren, dass es immer zuerst versucht Daten vom NASA-Server (in der angegebene Genauigkeit, Schalter --srtm) zu laden. Sollten für den Bereich keine Daten verfügbar sein, dann sollte versucht werden (automatisch oder über einen neuen Schalter) von ViewFinderPanoramas die Daten in der Auflösung von 3 Sekunden zu laden. Wurde zusätzlich der Schalter --viewfinder-mask=1 angegeben, dann wird eben noch versucht genauere Daten zu laden.


P.S.:
Muss dir noch für dein Programm ein Lob aussprechen. Ich habe vorher versucht die Karte mit srtm2osm zu erstellen. Das kommt damit aber überhaupt nicht zurecht und ist sehr langsam. Dein Programm leiste gute Arbeit in einer aktzeptablen Zeit.
Danke.
--AlexJoW 16:09, 2 February 2012 (UTC)

Einfach zwei Durchgänge. Durchgang eins: viewfindert3, dann die view3 Dateien die du benutzen willst, in den srtm3 Ordner schieben, und mit view1 einen neuen Durchgang starten. Es stimmt aber dass etwa "view1,view3,srtm3" als Angabe oder "view1,srtm1,srtm3" besser wären.--Extremecarver 02:12, 4 February 2012 (UTC)

Hallo, erst mal besten Dank für die schnelle Antwort. Ich hab das natürlich sofort ausprobiert. Hat aber leider nicht funktioniert. Vielleicht hab ich wieder was falsch gemacht. Den ersten Aufruf habe ich, nachdem die Daten herunter geladen waren, abgebrochen (gezippten Dateien wurden nicht komplett erstellt). Dann hab ich die heruntergeladenen Dateien wie beschrieben kopiert und den zweiten Aufruf gestartet, bevor ich die wenigen bereits erstellten gezippten Dateien gelöscht hatte. Das Ergebnis ist eine Karte, die nördlich des 61. Breitengrades nichts beinhaltet. Den ersten Aufruf habe ich abgebrochen, weil ich davon ausgegangen bin, dass es keinen Sinn macht, zwei Durchläufe komplett durchlaufen zu lassen, da meines Erachtens sich die Ergebnisse (gezippte Dateien) überschreiben würden und dann die Ids eventuell nicht mehr eindeutig sind. War das eventuell der Fehler? Oder müsste man nach dem Kopieren noch Anpassungen in den Index-Dateien vornehmen? --AlexJoW 13:30, 5 February 2012 (UTC)

Du kannst die Indexes löschen. Die Dateien müssen entpackt sein, dann werden sie nicht überschrieben. Aufpassen dass keine falschen Dateigrößen (gleich kaputte Downloads) vorkommen.--Extremecarver 08:35, 6 February 2012 (UTC)

Ich habe noch einmal einen Test durchgeführt:

HGT-Cache-Verzeichnis gelöscht.
1. Aufruf:
c:\python27\scripts\phyghtmap.exe -a -11:34:32:72 -s 10 -c 100,20 --start-node-id=2000000000 --start-way-id=1000000000 --corrx=0.0005 --corry=0.0005 --max-nodes-per-tile=2000000 --viewfinder-mask=3 --gzip=7
Dann die Dateien aus view3 nach srtm3 kopiert. Nichts gelöscht.
2. Aufruf:
c:\python27\scripts\phyghtmap.exe -a -11:34:32:72 -s 10 -c 100,20 --start-node-id=4000000000 --start-way-id=1100000000 --corrx=0.0005 --corry=0.0005 --max-nodes-per-tile=2000000 --viewfinder-mask=1 --gzip=7
Diesmal mit neuen Start-Ids, damit es keine Überschneidungen gibt
Wenn ich dann die Karte mit mkgmap erstelle, habe ich das Problem, dass es Bereiche gibt, für die z.B. Daten von view1 und srtm3 oder Daten von view3 und srtm3 vorliegen. Besonders der erste Fall ist nicht schön, da man in diesen Bereichen die Höhenlinien doppelt hat.
Also habe ich mir die Mühe gemacht und habe die von phyghtmap erstellten Dateien durchgesehen (ca. 2000 Stück). Dabei wurden alle "doppelten" manuell gelöscht und anschließend wieder die Karte erstellt. Diesen Durchlauf musste ich allerdings auch öfters machen, da ich immer wieder zu löschende Dateien übersehen hatte. Aber jetzt habe ich meine Höhenkarte von Gesamteuropa. War ein langer Kampf.

Deshalb auch noch mal meine Bitte an Panarchos das Programm zu erweitern, damit die manuelle Eingriffe nicht mehr nötig sind.--AlexJoW 16:17, 8 February 2012 (UTC)

In Version 1.43 gibt es die --source-Option: phyhtmap --source=view1,view3,srtm3 sollte das gewünschte Ergebnis erzielen. Das Problem der Lücke bei 60° nördlicher Breite kann ich allerdings leider nicht lösen.--Panarchos 09:34, 23 March 2012 (UTC)

Vielen Dank für die Erweiterung. Hab das natürlich gleich ausprobiert und muss sagen, dass es wunderbar funktioniert.

Mein Aufruf lautet nun:
phyghtmap.exe -a -11:34:32:72 -s 10 -c 100,20 --start-node-id=2000000000 --start-way-id=1000000000 --corrx=0.0005 --corry=0.0005 --max-nodes-per-tile=10000000 --source=view1,view3,srtm3 --gzip=7
Damit funktioniert es mit einem Aufruf, das Ergebnis ist eine Höhenlinienkarte für Gesamteuropa mit der besten Auflösung für jeden Bereich und eine Lücke am 60. Breitengrad gibt es auch nicht.
Nochmal vielen, vielen Dank.
--AlexJoW 16:59, 31 March 2012 (BST)

SRTM1 und USA funktioniert nicht

Mit den srtm1 Daten gibts in den USA noch größere Probleme bei phyghtmap. Hab kein einziges der geofabrik boundaries durchrechnen lassen können. Abbruch meist schon sehr schnell (und i.e. N38W077.hgt hab ich extra ein zweites mal runtergeladen um sicherzugehen dass es korrekt runtergeladen ist).:

Aufruf: phyghtmap --jobs=1 --osm-version=0.6 --step=20 --output-prefix=usne --line-cat=500,100 --srtm=1 --polygon=/home/contourlines/bounds/us-northeast.poly --max-nodes-per-tile=0 --max-nodes-per-way=250 --pbf (bzw mit --gzip=5 anstelle von --pbf was aber den selben Fehler bringt) ...... downloading .... hgt file hgt/SRTM1/N38W075.hgt: 3601 x 3601 points, bbox: (-75.00000, 38.00000, -74.00000, 39.00000) hgt file hgt/SRTM1/N38W076.hgt: 3601 x 3601 points, bbox: (-76.00000, 38.00000, -75.00000, 39.00000) hgt file hgt/SRTM1/N38W077.hgt: 3601 x 3601 points, bbox: (-77.00000, 38.00000, -76.00000, 39.00000) Traceback (most recent call last):

 File "/usr/local/bin/phyghtmap", line 9, in <module>
   load_entry_point('phyghtmap==1.41', 'console_scripts', 'phyghtmap')()
 File "/usr/local/lib/python2.7/site-packages/phyghtmap-1.41-py2.7.egg/phyghtmap/main.py", line 437, in main
   timestampString=output.timestampString))
 File "/usr/local/lib/python2.7/site-packages/phyghtmap-1.41-py2.7.egg/phyghtmap/main.py", line 272, in processHgtFile
   return ways # needed to complete file later

UnboundLocalError: local variable 'ways' referenced before assignment

Ist gefixt in Version 1.42.--Panarchos 09:23, 23 March 2012 (UTC)

Nich möglich SRTM1 und SRTM3 in einer Karte zu verwenden

Etwas ähnlich wie bei obiger Frage nach viewfinder1 und 3 Daten für eine Karte - was aber manuell lösbar ist, hier nochmal die bitte, den Aufruf etwas differenzierter zu gestalten. Für Kanada sind für einige Teile noch SRTM1 Daten verfügbar, für andere nicht. Leider ist es derzeit nicht möglich die SRTM1 Daten gleichzeit mit SRTM3 Daten zu benutzen (das umbenennen des Folders srtm1 in view1 und Aufruf mit viewfinder-mask=1 bringt nichts - die Daten werden einfach ignoriert).--Extremecarver 15:27, 20 February 2012 (UTC)

Ich habe jetzt die --source Option eingebaut (Version 1.43): phyghtmap --source=srtm1,srtm3 sollte das gewünschte Ergebnis erzielen.--Panarchos 09:25, 23 March 2012 (UTC)

Asien Output lässt sich nicht splitten mit mkgmap splitter.jar

phyghtmap --jobs=1 --osm-version=0.6 --step=25 --output-prefix=as2 --line-cat=1000,200 --source=view1,view3,srtm1,srtm3 --area=80:-11.01192:129.99999:83.20162 --max-nodes-per-tile=0 --max-nodes-per-way=250 --pbf java -Xmx5000m -jar splitter.jar --max-threads=4 --max-nodes=9000000 --mapid=75400000 --mixed --overlap=4000 as2*.osm.pbf Der Splitter erstellt nur ein einziges Output File, egal wie klein ich die max-nodes einstelle. Phyghtmap läuft sauber durch (braucht gut 6GB RAM, da 8 vorhanden sind aber keine Problem).--Extremecarver 12:32, 26 March 2012 (BST)

Problem liegt am mkgmap splitter. Es gibt einen Patch auf der Mailingliste der das Problem behebt.

Performance 1.43

While the initial check is much much faster, the actual work for me now takes about 5-10times longer. Is there a problem with my setup, or can other people verify this? It only happens if a polygon file is used. If giving coordinates speed/performance is unchanged. Extremecarver 17:09, 30 March 2012 (BST)

Ups seems to be strongly related to the polygon file. The bigger the worse. Could there be some bug related to larger polygon files sind 1.42?--Extremecarver 21:57, 1 April 2012 (BST)

Processing of large polygon files will be much faster in phyghtmap 1.44. I found a way to move the polygon membership test from the output data to the input data (i. e. the polygon membership test is done only in a few cases compared to the prior behaviour).--Panarchos 22:51, 11 September 2012 (BST)

other data sources?

Is there any way to process data from other sources? I have very good DEM data for the Piedmont from [3]. The data are in UTM32N and the resolution is 10X10m. I am a QGIS user and I've made very nice contours using QGIS, but the resulting contour data are in ESRI shapefiles, and so far I've found no easy way to get the shapefiles all the way to a garmin gmsupp.img file. So I was hoping to be able to go via OSM, but it's not so easy... I managed to convert the shapefiles to osm with ogr2osm, but I'm not sure what to do with the resulting osm file - when I run mkgmap with it, the resulting gmapsupp.img has nothing in it; obviously I would have to add attributes to it classifying the lines as contours, but so far I haven't worked out how to do that. Your tool looks like just the right thing, but it only takes SRTM data. And if I convert my data to mimick SRTM data, I sacrifice the detail I have.

Any hints? I've been looking for days now, and the fact that everyone seems to assume that anyone who wants to create contour lines would want to make them from SRTM data is driving me nuts. I think the cleanest way would be to import shapefiles and so leave users to create contours any way they like if they don't want to use the automatic creation from SRTM data. --Kfj 13:05, 28 May 2012 (BST)

I've found a way to pull shapefiles in so that I can use them in custom maps for my garmin. Ive described my bit of trickery in the wiki, maybe this can serve as inspiration:

http://wiki.openstreetmap.org/wiki/Contour_lines_from_other_Sources --Kfj 08:11, 4 June 2012 (BST)

Bugs in Debian Package phyghtmap_1.43-1_all.deb

Hi Panarchos,

I just now installed phyghtmap_1.43-1_all.deb at my Debian Testing and found two issues with it:

1.)

   File "/usr/bin/phyghtmap", line 5, in <module>
   from pkg_resources import load_entry_point
   ImportError: No module named pkg_resources

Problem: Dependency to python-pkg-resources is missing

Workaround: aptitude install python-pkg-resources

Will be fixed in phyghtmap 1.44.--Panarchos 22:51, 11 September 2012 (BST)

2.)

   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
   raise DistributionNotFound(req)
   pkg_resources.DistributionNotFound: phyghtmap==1.43

Problem: Distribution files are missing below /usr/lib/python2.7/dist-packages

Workaround:

    cd ../../python2.7/dist-packages/
    sudo ln -s ../../python2.6/dist-packages/phyghtmap
    sudo ln -s ../../python2.6/dist-packages/phyghtmap-1.43.egg-info/

This problem emerges from the fact that the current Debian stable (squeeze) doesn't come with python 2.7. When wheezy gets stable (which should soon be the case, the freeze is some time ago now), the package will be fixed.--Panarchos 22:51, 11 September 2012 (BST)

When possible can you fix the DEB package, or should I try to come up with a patch myself?

Thanks for the great tool, better than Monos :-)

Patch to phyghtmap 1.43-1 for support of hgt cache directory by command line option

I did not liked the way that the cache ./hgt/ directory is always created in the local directory, thus I implemented the command line option --hgtdir to specify the hgt file cache directory. Now I can use one directory across my file system.

Find the patch here: http://wiki.openstreetmap.org/wiki/User:Edamame/PatchPhyghtmapHgtDir

(Other formats I can of course create)

I also now created me a locally fixed deb package of the above mentioned errors. Let me know if you want to get the files.

BTW: Would you mind to move your source code to github.com or other collaboration platform? This would make it much easier to contribute patches.

Cheers! Edamame

Thanks alot. I will integrate the patch in 1.44.--Panarchos 22:51, 11 September 2012 (BST)

phyghtmap on Python 3.3

I'm trying to get phyghtmap to run on Python 3.3 (on Windows 7). So far, the only major problem I have discovered is in pbfUtil.py. In Python 3, binary and string data can no longer be mixed. I am fairly new to Python, but as far as I understand, pbfUtil will have to be converted to use bytearray instead of string. Do you have plans for such a conversion?

Best regards and thanks for the effort -- wolfbert

I just realized that the script will also output xml, and I can run it this way. -- wolfbert 17:25, 29 March 2014 (UTC)

Wunschliste: Option --download-only

Hallo,

wir verwenden phyghtmap um die Höhenlinien für die Freizeitkarte (http://freizeitkarte-osm.de/) zu bauen. Zusätzlich hole ich mir dann meistens noch die hgt Dateien aus dem entsprechenden Verzeichnis um für Qlandkarte GT und andere Applikationen die DEM Daten für Hillshading zusammenzustellen. Oft wäre ich froh, wenn ich phyghtmap verwenden könnte um

1. die hgt Dateien für ein poly (*.poly) runterzuladen
2. anzuzeigen welche hgt Dateien nötig sind für das gewünschte poly
3. das ganze ohne wirklich Höhenkurven zu rechnen (das läuft in einem anderen Schritt ab)

Im Moment behelfe ich mir so, dass ich als output file ein directory/file angebe, das nicht existiert und phyghtmap mit einem stack trace umfällt.... ist aber unschön. Optimaler wäre es (nicht nur für mich, denke ich), wenn man mit einer Option --download-only den Download Prozess (Netzwerkintensiv) und den Höhenkurven-Berechnungsprozess (CPU intensiv) voneinander trennen könnte. ... und ich hätte so die Möglichkeit phyghtmap ein klein wenig zweckentfremdet für meine zusätzliche Anwendung einzusetzen.

Gruss und danke im Voraus

keenonkites

ENGLISH (short version): I'd like to have a new option --download-only that doesn't process any contour lines but just prepares all needed hgt files to cover a specific given poly file. This way the network intensive download can be separated from the calculations and it's easy to use phyghtmap also for downloading and preparing hgt files for hillshading for applications like QLandkarte GT and similar.

Ist implementiert in Version 1.61. --Panarchos (talk) 17:42, 22 February 2015 (UTC)

Bin etwas spät dran aber trotzdem vielen Dank... super -- keenonkites 13.4.2015

ERROR Messages

I am getting the following RuntimeError: maximum recursion depth exceeded while calling a Python object
with the following Parameters under Windows 7 with Python 2.7 - 64 bit

• phyghtmap.exe --area=6:43:7:46 --source=view1,srtm1
• phyghtmap.exe --area=7:43:8:44 --source=view1,srtm1
• phyghtmap.exe --area=19:49:21:50 --source=view1,srtm1
• phyghtmap.exe --area=-25:63:-13:64 --source=view3

I'm not getting any error with srtm1 or srtm3 data. The last area is north than 60°, so there is no srtm3 data here.
Can somebody please try to reproduce these errors under Windows since they are not reproduceable unter Linux. --WalterSchloegl (talk) 22:49, 12 February 2015 (UTC)

I suppose that this problem arises from the fact that the default recursion limit in python is platform-dependent. On my linux installation it's 1000. What is it on a Windows machine? (in a python shell, say: "import sys; sys.getrecursionlimit()") This basically happens because there is much data in these areas and phyghtmap recursively tries to chop the areas into smaller pieces until a maximum number of nodes per each tile is not exceeded. This can actually be circumvented by either setting this number to a higher value using the --max-nodes-per-tile option (see the output of phyghtmap.exe --help for more info) or by splitting the phyghtmap calls like e. g. phyghtmap.exe --area=6:43:6.5:46 --source=view1,srtm1 and phyghtmap.exe --area=6.5:43:7:46 --source=view1,srtm1. --Panarchos (talk) 17:19, 22 February 2015 (UTC)

Wunschliste: Geotiff input

Hallo, Vielen Dank für das nützliche Tool. Momentan erzeugen wir mit phyghtmap die Höhenlinien für die Garmin-Karte der OpenTopoMap. Die Höhenlinien der Rasterkarte entstehen derzeit mit gdal_contour als einzelne Shapefiles. Das ist alles andere als praktisch und im Zuge der Vorbereitung auf weltweite Abdeckung möchten wir eine bessere Toolchain entwickeln. Da selbst die SRTM-Daten von viewfinderpanoramas noch Fehlpixel besitzen, jagen wir diese erst durch gdal_fillvoids.py und mergen sie zu einer großen Geotiff. Von dieser ausgehend erzeugen wir eine Umprojektion (mit Interpolation), ein Relief und die Schummerung. Perfekt wäre es nun, wenn phyghtmap auch Geotiffs lesen könnte. Die Ausgabedatei als pbf lässt sich nun leicht in eine Datenbank importieren und für die Zukunft abspeichern.

Wäre es möglich, dass phyghtmap eine geotiff-Eingabe bekommt?

Vielen Dank und schöne Grüße

Stefan von OpenTopoMap --Derstefan (talk) 11:50, 8 March 2015 (UTC)

SRTM1 v3 Europe not downloading

Hi Parnarchos - phyghtmap is not able to download the SRTM1 v3 data for Europe as I tried it out - are you sure it grabs the right sources? As to my understanding except for the middle east/Northeast Africa worldwide SRTM1 data is now available (<60°N of course).--Extremecarver (talk) 08:40, 13 May 2015 (UTC)

Wunschliste: lokale hgt files kombiniert mit Schnitt via Polygon

Im Moment werden hgt files, die auf der Kommandozeile mitgegeben werden, ignoriert, sobald ich ein polygon für den korrekten Zuschnitt angebe. Hat das einen speziellen Grund oder könnte das in einer nächsten Version angepasst werden ? Jegliches spätere Zuschneiden von den generierten Höhenlinien führt zu Problemen wie Artefakte und ähnliches. Grund dieser Anfrage/dieses Wunsches: es gibt für immer mehr Gegenden/Länder höherauflösende freie Geländemodelle als die direkt unterstützten Quellen (view1 view3 srtm1 srtm3), meistens in Form von hgt Dateien. Um diese einfach und direkt verarbeiten zu können, wäre diese Kombination von Argumenten und Optionen nötig.

At the moment manually feed hgt files are ignored as soon as a polygon is given for precise cutting of the contour lines. Is there a special reason for this or could this be a possible combination for a future version ? If I have to cut contour lines after generation it always leads into troubles like artefacts. Reason for this request is that there are more and more sources providing hgt files with higher precision than the directly supported sources. To have the same easy and direct workflow the combination of manually feed hgt files and a cutting polygon would be needed.

Nachfrage / Inquiry (2016-11-30)

Gibt's eine chance, dass das in die nächste Version implementiert wird ? Wäre super...

Any chance that this request gets implemented into the next version ? Would be absolutely great

Errors Messages

I am getting two different error messages when running phyghtmap 1.80 with following options:

 phyghtmap \
 --step=20 \
 --line-cat=200,100 \
 --polygon=./POLY/Europe.poly \
 --o5m \
 --no-zero-contour \
 --jobs 8 \
 --source=view1,view3 \
 --hgtdir=./Elevation/hgt \
 --output-prefix=contour 

Most tiles are running fine, however sometimes this message appears:

 hgt file ./Elevation/hgt/VIEW3/N61E018.hgt: 1201 x 1201 points, bbox: (18.00000, 61.00000, 19.00000, 62.00000)
 Computing ./Elevation/hgt/VIEW1/N60E018.hgt
 Traceback (most recent call last):
 File "/usr/bin/phyghtmap", line 11, in <module>
   load_entry_point('phyghtmap==1.80', 'console_scripts', 'phyghtmap')()
 File "/usr/lib/python2.7/dist-packages/phyghtmap/main.py", line 644, in main
   queue.process()
 File "/usr/lib/python2.7/dist-packages/phyghtmap/main.py", line 619, in process
   self.processMultiOutput()
 File "/usr/lib/python2.7/dist-packages/phyghtmap/main.py", line 606, in processMultiOutput
   expectedNumOfWays, expectedNumOfNodes = self._forkOneMultiOutput()
 File "/usr/lib/python2.7/dist-packages/phyghtmap/main.py", line 590, in _forkOneMultiOutput
   checkPoly=checkPoly)
 File "/usr/lib/python2.7/dist-packages/phyghtmap/main.py", line 420, in processHgtFile
   output.writeWays(ways, opts.startWayId)
 File "/usr/lib/python2.7/dist-packages/phyghtmap/o5mUtil.py", line 174, in writeWays
   self.writeWay(ways[0], idDelta=startWayId, first=True)
 IndexError: list index out of range

And sometimes this one:

 hgt file ./Elevation/hgt/VIEW1/N64E017.hgt: 3601 x 3601 points, bbox: (17.00000, 64.00000, 18.00000, 65.00000)
 Panic: Didn't work: ./Elevation/hgt/VIEW3/N61E018.hgt

A--Chasuny hints what to do? And thanks a lot for this program, it really is simple and straightforward to use! --[[User:Chasu|Chasu (talk) 07:36, 16 August 2017 (UTC)

phyghtmap 2.00 - error in NASASRTMUtil.py

phyghtmap works fine for me - but the downloader seems to be broken - i'm on current Ubuntu LTS. Below is the error log:

phyghtmap --earthdata-user=test  --earthdata-password=Test  --jobs=1 --step=20 --no-zero-contour --void-range-max=-420 --output-prefix=bhutan --line-cat=500,100 --polygon=/home/contourlines/bounds/bhutan.poly --start-node-id=10000000 --start-way-id=10000000 --source=view1,view3,srtm1,srtm3 --max-nodes-per-tile=0 --max-nodes-per-way=230 --pbf
checking if area N28E088 intersects with polygon ... no
checking if area N27E090 intersects with polygon ... yes
checking if area N28E092 intersects with polygon ... no
N26E088: trying view1 ...
N26E088: trying view3 ...
N26E088: downloading file http://viewfinderpanoramas.org/dem3/G45.zip to hgt/VIEW3/G45.zip ...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 624, in downloadAndUnzip
FileNotFoundError: [Errno 2] No such file or directory: 'hgt/VIEW3/N26E088.hgt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 633, in downloadAndUnzip
FileNotFoundError: [Errno 2] No such file or directory: 'hgt/VIEW3/G45.zip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/phyghtmap", line 9, in <module>
    load_entry_point('phyghtmap==2.0', 'console_scripts', 'phyghtmap')()
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/main.py", line 629, in main
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 692, in getFiles
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 682, in getFile
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 645, in downloadAndUnzip
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 616, in downloadToFile
  File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.0-py3.5.egg/phyghtmap/NASASRTMUtil.py", line 607, in downloadToFile_Simple
AttributeError: module 'ssl' has no attribute 'PROTOCOL_TLS'

--Extremecarver (talk) 12:48, 21 November 2017 (UTC)

Multi-Threading not working

It looks like pyghtmap uses only 1 thread form my CPU (--jobs=4 is defined). Is this a Bug? Or do i something wrong? :-)

Locally sourced folder

As has been mentioned already, it would really be good to have the option to include other sources. The best concept would be as option for Soures Local1, Local3, View1, View3, SRTM1, SRTM3 that way dirty hacking around in the *hgtindex*.txt files would not be needed anymore. So you create a folder called LOCAL1 for all 1" hgt/tiff files you want to provide, LOCAL3 for all 3" files. The index files should be created before just like for VIEW and SRTM but based on folder content.

Right now I worked around this by changing the viewfinderHGTIndex_1.hgt and adding all local files into it. Not really sure how well this is working. With OpenDEM.info providing the EU Copernicus 1" DEM data in .hgt format there is a very good alternative source that is missing, plus several other rather local sources. Replacing the files in SRTM1v3.0 is not very workable right now with the earthexplorer servers replying super slow - so you basically cannot use SRTM1 right now if you have not downloaded all the data for the required polygon already in the past. --Extremecarver (talk) 16:33, 1 May 2018 (UTC)

Bug with SRTM1v3.0 geotiff input files

None of the geotiff files seems to be usable due to some bug?

Computing hgt/SRTM1v3.0/S47E167.tif tif file hgt/SRTM1v3.0/S47E167.tif: 3601 x 3601 points, bbox: (167.00000, -47.00000, 168.00000, -46.00000) /usr/local/lib/python3.5/dist-packages/phyghtmap-2.10-py3.5.egg/phyghtmap/hgt.py:735: RuntimeWarning: invalid value encountered in remainder /usr/local/lib/python3.5/dist-packages/phyghtmap-2.10-py3.5.egg/phyghtmap/hgt.py:737: RuntimeWarning: invalid value encountered in remainder Computing hgt/SRTM1v3.0/S48E167.tif tif file hgt/SRTM1v3.0/S48E167.tif: 3601 x 3601 points, bbox: (167.00000, -48.00000, 168.00000, -47.00000) /usr/local/lib/python3.5/dist-packages/phyghtmap-2.10-py3.5.egg/phyghtmap/hgt.py:735: RuntimeWarning: invalid value encountered in remainder /usr/local/lib/python3.5/dist-packages/phyghtmap-2.10-py3.5.egg/phyghtmap/hgt.py:737: RuntimeWarning: invalid value encountered in remainder

and so on for all files downloaded using new-zealand.poly by geofabrik.

Oh and in general it would be nice if phyghtmap could download the SRTM data in hgt instead of geotiff as it used to be. mkgmap needs hgt for dem processing, and conversion from hgt to geotiff is very easy, while it is difficult the other way around. They are also downloadable as hgt - seems it connects to earthdata not earthexplorer though: http://dwtkns.com/srtm30m/ I tried and failed to adapt nasasrtmutil.py to the earthdata source... Earthexplorer source for zipped SRTM1v3.0 is http://e4ftl01.cr.usgs.gov//MODV6_Dal_D/SRTM/SRTMGL1.003/2000.02.11/{:s}.SRTMGL1.hgt.zip--Extremecarver (talk) 22:53, 4 May 2018 (UTC)

Same problems for me running on Windows. I have also succeeded in adapting to update indices but not download hgt files from this Earthdata source.

--Lcdetwiler (talk) 01:19, 11 September 2018 (UTC)

I have now succeeded in downloading from earthdata source http://e4ftl01.cr.usgs.gov//MODV6_Dal_D/SRTM/SRTMGL1.003/2000.02.11/{:s}.SRTMGL1.hgt.zip by reverting to earthdata login and modifying downloadToFile_SRTMv3(url, filename) from phyghtmap v2.0 with the instructions from https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+Python

--Lcdetwiler (talk) 03:05, 13 September 2018 (UTC)

Phyghtmap on Python 3 in Windows

I have been using Phyghtmap successfully for a year now with Python 2.7 in Windows. I have now upgraded to Python 3.6.4 and cannot get Phyghtmap to work anymore even after reinstalling all the dependencies. I have: phyghtmap 2.10 python 3.6.4 setuptools 39.1.0 matplotlib 2.2.2 beautifulsoup4 4.6.0

The error I get when I try run phyghtmap with any parameters is always the same, Traceback (most recent call last):

 File "C:\Python36-32\Scripts\phyghtmap-script.py", line 11, in <module>
   load_entry_point('phyghtmap==2.10', 'console_scripts', 'phyghtmap')()
 File "C:\Python36-32\lib\site-packages\pkg_resources\__init__.py", line 480, in load_entry_point
   return get_distribution(dist).load_entry_point(group, name)
 File "C:\Python36-32\lib\site-packages\pkg_resources\__init__.py", line 2691, in load_entry_point
   return ep.load()
 File "C:\Python36-32\lib\site-packages\pkg_resources\__init__.py", line 2322, in load
   return self.resolve()
 File "C:\Python36-32\lib\site-packages\pkg_resources\__init__.py", line 2328, in resolve
   module = __import__(self.module_name, fromlist=['__name__'], level=0)
 File "C:\Python36-32\lib\site-packages\phyghtmap-2.10-py3.6.egg\phyghtmap\main.py", line 17, in <module>
 File "C:\Python36-32\lib\site-packages\phyghtmap-2.10-py3.6.egg\phyghtmap\hgt.py", line 10, in <module>

ImportError: cannot import name '_cntr'

It looks like the problem is that my version of matplotlib does not have the _cntr module anymore.

What versions of the dependencies (matplotlib, bs4 etc) are you using so that I can try run on the same versions?

The versions are all listed above. It really seems horrible now under windows with python 3.6.5. I tried installing everything using pip install XXXXX but I don't manage to actually get all the dependencies working for pip :: install gdal. I'm not sure anymore how I got the above setup working.

I've now tried using Python 3.7 beta 4. Now also numpy and matplotlib are not installing anymore with pip install XXXX. Therefore I installed, gdal, matplotlib and numpy as a wheel from https://www.lfd.uci.edu/~gohlke/pythonlibs/

using pip install XXX.whl - e.g. pip install matplotlib-2.2.2-cp37-cp37m-win_amd64.whl.

However also with the new stack all based on Python 3.7 the error message is more or less the same:

C:\OpenMTBMap\phyghtmap>"C:\Program Files\Python37\Scripts\phyghtmap.exe"

Traceback (most recent call last):

 File "C:\Program Files\Python37\Scripts\phyghtmap-script.py", line 11, in <module>
   load_entry_point('phyghtmap==2.10', 'console_scripts', 'phyghtmap')()
 File "C:\Program Files\Python37\lib\site-packages\pkg_resources\__init__.py", line 480, in load_entry_point
   return get_distribution(dist).load_entry_point(group, name)
 File "C:\Program Files\Python37\lib\site-packages\pkg_resources\__init__.py", line 2693, in load_entry_point
   return ep.load()
 File "C:\Program Files\Python37\lib\site-packages\pkg_resources\__init__.py", line 2324, in load
   return self.resolve()
 File "C:\Program Files\Python37\lib\site-packages\pkg_resources\__init__.py", line 2330, in resolve
   module = __import__(self.module_name, fromlist=['__name__'], level=0)
 File "C:\Program Files\Python37\lib\site-packages\phyghtmap-2.10-py3.7.egg\phyghtmap\main.py", line 17, in <module>
 File "C:\Program Files\Python37\lib\site-packages\phyghtmap-2.10-py3.7.egg\phyghtmap\hgt.py", line 10, in <module>
 ImportError: cannot import name '_cntr' from 'matplotlib' (C:\Program Files\Python37\lib\site-packages\matplotlib\__init__.py)

New Option --simplifyContoursEpsilon=0.0002 not working/broken

EDIT November 2018 - In Ubuntu 18.04 LTS it now works without error.

Ubuntu 16.04 LTS - running: phyghtmap --jobs=8 --polygon=/home/contourlines/bounds/$spain.poly --step=$20 --simplifyContoursEpsilon=0.0002 --no-zero-contour --void-range-max=-420 --output-prefix=spain --line-cat=500,100 --start-node-id=10000000 --start-way-id=10000000 --source=view1,srtm1,view3,srtm3 --max-nodes-per-way=230 --max-nodes-per-tile=0 --pbf

Computing hgt/VIEW1/N40E004.hgt hgt file hgt/VIEW1/N40E004.hgt: 3601 x 3601 points, bbox: (4.00000, 40.00000, 5.00000, 41.00000), checking polygon borders Traceback (most recent call last):

 File "/usr/local/bin/phyghtmap", line 9, in <module>
   load_entry_point('phyghtmap==2.20', 'console_scripts', 'phyghtmap')()
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 680, in main
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 653, in process
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 552, in processSingleOutput
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 525, in _forkOneSingleOutput
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 425, in processHgtFile
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 869, in countNodes
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 869, in <listcomp>
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 436, in trace
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 384, in simplifyPath

AttributeError: 'list' object has no attribute 'shape' Traceback (most recent call last):

 File "/usr/local/bin/phyghtmap", line 9, in <module>
   load_entry_point('phyghtmap==2.20', 'console_scripts', 'phyghtmap')()
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 680, in main
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 653, in process
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 552, in processSingleOutput
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 536, in _forkOneSingleOutput
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 536, in <listcomp>

ValueError: invalid literal for int() with base 10:

Thought maybe only applies to 1" data - but also with VIEW3 data same error: Computing hgt/VIEW3/N40E004.hgt hgt file hgt/VIEW3/N40E004.hgt: 1201 x 1201 points, bbox: (4.00000, 40.00000, 5.00000, 41.00000), checking polygon borders Traceback (most recent call last): ..........................................

If I run with jobs=1 - then the error message changes: hgt file hgt/VIEW3/N42W010.hgt: 1201 x 1201 points, bbox: (-10.00000, 42.00000, -9.00000, 43.00000), checking polygon borders Traceback (most recent call last):

 File "/usr/local/bin/phyghtmap", line 9, in <module>
   load_entry_point('phyghtmap==2.20', 'console_scripts', 'phyghtmap')()
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 693, in main
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 473, in processHgtFile
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/main.py", line 390, in writeNodes
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/pbfUtil.py", line 411, in writeNodes
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 436, in trace
 File "/usr/local/lib/python3.5/dist-packages/phyghtmap-2.20-py3.5.egg/phyghtmap/hgt.py", line 384, in simplifyPath

AttributeError: 'list' object has no attribute 'shape'

cannot convert float NaN to integer

I have self made python3-phyghtmap-2.20 rpm installed on fedora 28i system and I get the following error:

Traceback (most recent call last):

 File "/usr/bin/phyghtmap", line 11, in <module> load_entry_point('phyghtmap==2.20', 'console_scripts', 'phyghtmap')()
 File "/usr/lib/python3.6/site-packages/phyghtmap/main.py", line 696, in main checkPoly=checkPoly))
 File "/usr/lib/python3.6/site-packages/phyghtmap/main.py", line 489, in processHgtFile elevations, output.timestampString, opts)
 File "/usr/lib/python3.6/site-packages/phyghtmap/main.py", line 390, in writeNodes return pbfUtil.writeNodes(*args, **kwargs)
 File "/usr/lib/python3.6/site-packages/phyghtmap/pbfUtil.py", line 414, in writeNodes newNodes, newWays = _makeNodesWays(contourList, elevation, IDCounter)
 File "/usr/lib/python3.6/site-packages/phyghtmap/pbfUtil.py", line 396, in _makeNodesWays newNodes, nodeRefs = _makePoints(path, elevation, IDCounter)
 File "/usr/lib/python3.6/site-packages/phyghtmap/pbfUtil.py", line 383, in _makePoints nodes.append((int(lon*NANO), int(lat*NANO)))
 ValueError: cannot convert float NaN to integer

Is it a bug of phyghtmap or in some library or is it some incompatibility?

Maybe a bug in phyghtmap

I have had the same error running under Windows - seems to be due to NaN's left behind by the trace() function in hgt.py if you are using

matplotlib version >= 2.0.0

--Lcdetwiler (talk) 01:08, 11 September 2018 (UTC)

tif not recognized as a supported file format.

phyghtmap is failing for me. Output below. Running on linux, the latest version of phytgmap.

phyghtmap -a -106.124:38.492:-105.970:38.562 N38W107: trying srtm3v3.0 ... N38W107: using file hgt/SRTM3v3.0/N38W107.tif. N38W106: trying srtm3v3.0 ... N38W106: using file hgt/SRTM3v3.0/N38W106.tif. ERROR 4: `hgt/SRTM3v3.0/N38W107.tif' not recognized as a supported file format. Traceback (most recent call last):

 File "/usr/lib/python3/dist-packages/phyghtmap/hgt.py", line 533, in initAsGeotiff
   geoTransform = g.GetGeoTransform()

AttributeError: 'NoneType' object has no attribute 'GetGeoTransform'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

 File "/usr/bin/phyghtmap", line 11, in <module>
   load_entry_point('phyghtmap==2.20', 'console_scripts', 'phyghtmap')()
 File "/usr/lib/python3/dist-packages/phyghtmap/main.py", line 696, in main
   checkPoly=checkPoly))
 File "/usr/lib/python3/dist-packages/phyghtmap/main.py", line 399, in processHgtFile
   checkPoly, opts.voidMax, opts.contourFeet)
 File "/usr/lib/python3/dist-packages/phyghtmap/hgt.py", line 486, in __init__
   self.initAsGeotiff(corrx, corry, polygon, checkPoly, voidMax)
 File "/usr/lib/python3/dist-packages/phyghtmap/hgt.py", line 549, in initAsGeotiff
   self.lonIncrement = geoTransform[1]

UnboundLocalError: local variable 'geoTransform' referenced before assignment

Voids in view3 data make phyghtmap to barf

I was building contour lines for Israel from view3 using phyghtmap. It failed to make contours for half of the country.

phyghtmap version v2.20-1 (.deb), matplotlib Debian 2.0.0.

So I took a look. I could see that several hgt files failed. I found out the following:

- the failed files contained voids (0x8000).
- phyghtmap masked the voids
- QuadContourGenerator processed OK but resulted in some paths containing nans.
- in hgt.py _makePoints in nodes.append int() failed for a nan thus causing
  exception aborting handling of these hgt files

Initial fix was skipping the nans in the paths (and also skipping the resulting empty paths).

However I noticed that the voids in the hgt files were only very few and were isolated in the middle of good correct elevation data. So I added interpolation over the voids (from scipy.interpolate import griddata) right after reading the file. This resulted having no nans in the paths and also visually very good contour lines.

Diffs below

Well it seems I don't have permissions to upload. How can I get it to you? Still can't upload.

phyghtmap on MacOS

I confirm that phyghtmap 2.21 does work on MacOS 10.14 Mojave with Deployment Manager (EDM). Some dependencies are not listed in the notes for MacOS X users, though.

First, please notice that you should create a python 3 environment with:

edm install --version 3

Then, regarding dependencies, I had to install matplotlib, numpy, gdal, and beautifulsoup4 (corresponding to bs4), that are listed on the website, as well as libjpeg and libtiff, that are not listed:

$ edm install libjpeg
$ edm install libtiff

I also needed to set the GDAL_DATA environment variable, inside my edm environment:

(edm)bash-3.2$ export GDAL_DATA=$HOME/.edm/envs/edm/share/gdal

Obviously, phyghtap has to be executed from inside the edm environment:

(edm)bash-3.2$ phyghtmap -a 6.83:44.99:6.90:45.04 (...)

--solitone (talk) 10:41, 26 February 2019 (UTC)

SRTM Downloads not working (phyghtmap 2.21)

Hello Panarchos,

I'm running phyghtmap 2.21 on Ubuntu 16.04.6 LTS. I can download and process the view1/view3 date without any problems. But no matter what combinations of srtm sources and versions I'm trying I'm unable to download anything, it always gives back: N48E015: no file found on server. (changing coordinate tag, obviously). I have tried with multiple working logins on earthexplorer, even created a new one following the description in the tool.

Did the source to download srtm1/srtm3 change once more ? Is there a way to download them manually (somwhere) and put provide them to phyghtmap for further processing ?

Thanks and cheers

keenonkites

SRTM Downloads: semi-solution

You could download SRTM1/3 files via new USGS "download" application. Login to you UGSG Account ("GeoExplorer" account) at https://earthexplorer.usgs.gov/, select area on map, select "Data Sets | SRTM | SRTM 1 Arc-second Global" or "SRTM void filled"), click "Results". Then you can add all results (there will be multiple pages, pay attention to pagination controls!) into "Basket". It is free, so don't be scared by Basket and/or Order words. Get "Bulk Download" application at https://dds.cr.usgs.gov/bulk/ (it is Java application and should work on all systems), login into your account in application and you will see Order (effectivly download queue). Click "Download" and wait.

You should rename files (format is obvious) and put it into phyghtmap cache dir (hgt/SRTM1v3.0 ot hgt/SRTM3v3.0). Rebuild phyghtmap index.

phyghtmap for MacOSX

Following works for me with current MacOSX:

sudo python3 -mpip install bs4 matplotlib

sudo python3 setup.py install

phyghtmap -s5 *.hgt

Can you add it on this site? http://katze.tfiu.de/projects/phyghtmap/#MacOSXDependencies

B-Spline Interpolation

Would it be possible to add B-Spline interpolation into phyghtmap? OSM pbf data for contourlines created with gnuplot and b-spline interpolation looks much cleaner and in Garmin format also consumes less size. The contourlines map of kleineisel.de for example have this. The difference is really big when zoomed in very close in mountaineous areas. --simplifyContoursEpsilon=? helps a bit too - but is super slow. Not sure if gnuplot would be faster too. https://wiki.openstreetmap.org/wiki/Srtm2osm_perl is not working anymore - but that script was editable to also use b-spline interpolation with gnuplot and it required ASC data as input... I'm not sure if there is another way instead of gnuplot to use b-spline interpolation instead of linear interpolation with phyghtmap.--Extremecarver (talk) 02:57, 26 April 2021 (UTC)

Download retry option

Please add an option for retrying a download when it times out:

S10W151: downloading file https://earthexplorer.usgs.gov/download/5e83a3efe0103743/SRTM1S10W151V3/EE to /root/mapsforge/hgt/SRTM1v3.0/S10W151.tif ... Traceback (most recent call last):

 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 718, in downloadAndUnzip_Tif
   os.stat(saveFilename)

FileNotFoundError: [Errno 2] No such file or directory: '/root/mapsforge/hgt/SRTM1v3.0/S10W151.tif'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

 File "/usr/bin/phyghtmap", line 33, in <module>
   sys.exit(load_entry_point('phyghtmap==2.23', 'console_scripts', 'phyghtmap')())
 File "/usr/lib/python3/dist-packages/phyghtmap/main.py", line 662, in main
   opts.srtmCorrx, opts.srtmCorry, opts.dataSource)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 804, in getFiles
   saveFilename = getFile(area, source)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 794, in getFile
   return downloadAndUnzip(url, area, source)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 702, in downloadAndUnzip
   return downloadAndUnzip_Tif(url, area, source)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 721, in downloadAndUnzip_Tif
   downloadToFile(url, saveFilename, source)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 697, in downloadToFile
   return downloadToFile_SRTMv3(url, filename)
 File "/usr/lib/python3/dist-packages/phyghtmap/NASASRTMUtil.py", line 681, in downloadToFile_SRTMv3
   res = opener.open(url)
 File "/usr/lib/python3.6/urllib/request.py", line 532, in open
   response = meth(req, response)
 File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
   'http', request, response, code, msg, hdrs)
 File "/usr/lib/python3.6/urllib/request.py", line 564, in error
   result = self._call_chain(*args)
 File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
   result = func(*args)
 File "/usr/lib/python3.6/urllib/request.py", line 756, in http_error_302
   return self.parent.open(new, timeout=req.timeout)
 File "/usr/lib/python3.6/urllib/request.py", line 526, in open
   response = self._open(req, data)
 File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
   '_open', req)
 File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
   result = func(*args)
 File "/usr/lib/python3.6/urllib/request.py", line 1368, in https_open
   context=self._context, check_hostname=self._check_hostname)
 File "/usr/lib/python3.6/urllib/request.py", line 1328, in do_open
   r = h.getresponse()
 File "/usr/lib/python3.6/http/client.py", line 1373, in getresponse
   response.begin()
 File "/usr/lib/python3.6/http/client.py", line 311, in begin
   version, status, reason = self._read_status()
 File "/usr/lib/python3.6/http/client.py", line 280, in _read_status
   raise RemoteDisconnected("Remote end closed connection without"

http.client.RemoteDisconnected: Remote end closed connection without response

Flipping / Reversing Coordinates with GDal >= 3

After a day of frustration I have discovered that when using it with GDal version 3 or above phyghtmap will flip the coordinates, using lat for lon and lon for lat. I have described the issue on the GIS stackexchange and I was also able to fix it by patching hgt.py in 3 places (the file is posted to github as a gist). It would be cool if a new version would be released with some sort of fix. Thanks!

--Kitsched (talk) 19:41, 11 February 2022 (UTC)

(Windows) ValueError: too many values to unpack

I can't get the script to work, I'm on Windows, I have all the dependencies fine, but it still doesn't work, I try with smaller sections, but it's the same. Thank you in advance for your help.

1. I find info in the net [[4]]

C:\Program Files\Python310\lib\site-packages\phyghtmap\hgt.py:275: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a 
list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when 
creating the ndarray.
 return [numpy.array(tmpList), ]
Traceback (most recent call last):
 File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
   return _run_code(code, main_globals, None,
 File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
   exec(code, run_globals)
 File "C:\Program Files\Python310\Scripts\phyghtmap.exe\__main__.py", line 7, in <module>
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\main.py", line 695, in main
   ways.extend(processHgtFile(hgtDataFileName, opts, output,
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\main.py", line 488, in processHgtFile
   opts.startId, ways = writeNodes(output, contourData,
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\main.py", line 390, in writeNodes
   return pbfUtil.writeNodes(*args, **kwargs)
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\pbfUtil.py", line 414, in writeNodes
   newNodes, newWays = _makeNodesWays(contourList, elevation, IDCounter)
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\pbfUtil.py", line 396, in _makeNodesWays
   newNodes, nodeRefs = _makePoints(path, elevation, IDCounter)
 File "C:\Program Files\Python310\lib\site-packages\phyghtmap\pbfUtil.py", line 381, in _makePoints
   for lon, lat in path:
ValueError: too many values to unpack (expected 2)

Ich habe einige Anpassungen gemacht, um das Problem zu beheben

Ich hoffe es ist ok, wenn ich deutsch schreibe. Da ich das gleiche Problem hatte, habe ich die Sourcen debuged und konnte so (hoffentlich) ein paar der Probleme fixen. Zu meiner Umgebung: Win 10, Python 3.10, die benötigten packages sind in der aktuellsten Version installiert.

osmUtil.py: Zeile 118
	ids, content = [], []
	try:
		for lon, lat in path:
...
	except:
		return []
	return ids

pbfUtil.py: Zeile 380
	ids, nodes = [], []
	try:
		for lon, lat in path:
...
	except:
		return [], []
	return nodes, ids

hgt.py: Zeile 422
		if mplversion >= "3.0.0":
			rawPaths = self.Cntr.create_contour(elevation)
			rawPaths = rawPaths[0] # this helps, if it is correct? i dont know what create_contour is doing ...
		elif mplversion >= "2.0.0":
			rawPaths = self.Cntr.create_contour(elevation)

Wenn jemand die Ursache in hgt.py kennt / versteht, wäre ich für Tipps dankbar.

I also got the "ValueError: too many values to unpack" error when using phyghtmap on an Apple M1 Mac

The fix show above didn't work for me, it just gave me a different error.

It seems that phyghtmap uses an undocumented module in matplotlib called either _cntr or _contour depending on the version of matplotlib. Unfortunately in the latest versions of matplotlib (I was using 3.5.3) the contour drawing module has changed again and doesn't work anymore.

For me the solution was to downgrade matplotlib to 2.2.0 and phyghtmap now works fine. However I have not investigated which versions of matplotlib between 2.2.0 and 3.5.3 work.

For info I'm using python 3.9, beautifulsoup4 4.11.1, setuptools 63.4.3, lxml 4.9.1, matplotlib 2.2.0, GDAL 3.5.1, all the latest versions except matplotlib.

--Cboothroyd (talk) 08:55, 29 August 2022 (UTC)

To complete Cboothroyd's update (thanks, it led me to the root cause), the "ValueError: too many values to unpack" issue is actually due to some non-backward compatible changes done in matplotlib._contour in branch 3.5.x: https://github.com/matplotlib/matplotlib/commit/178012d136b2a34e22489658108d4de685150a36#diff-b3f136a845ae5b7383fbc2be280c64af12352a6583ace9658cc4f6e2479df4a3

This internal module has anyway been removed in matplotlib 3.6 in favor of contourpy: https://github.com/matplotlib/matplotlib/commit/20c5af3e15724d15e78843938dd836c8a9142a75

I've created a fork (hoping it will ease collaboration) with the integration of this change: https://github.com/agrenott/pyhgtmap. I've imported all the available historical versions from the download page. I've also added some basic unit tests and CI/CD.

@Panarchos please feel free to contact me if you have any objection.

--Agrenott (talk) 22:41, 3 March 2023 (UTC)

Ubuntu 22.04 phyghtmap==2.10 distribution was not found

Ich hab das .deb von http://katze.tfiu.de/projects/phyghtmap/#DebianPackage heruntergeladen und installiert, und bekomme folgenden Fehler:

/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.16.0-unknown is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 2.22.1ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 573, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 891, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (phyghtmap 2.23 (/usr/lib/python3/dist-packages), Requirement.parse('phyghtmap==2.10'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/phyghtmap", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3267, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3241, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3279, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 575, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 588, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 777, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'phyghtmap==2.10' distribution was not found and is required by the application

Mit 18.04 gings noch. Hat jemand evtl eine Idee?

Ubuntu 22.04 phyghtmap==2.23 failure on compiling and subsequently phyghtmap will not run

root@AX41-FSN1 /home/contourlines/phyghtmap-2.23 # sudo python3 setup.py install running install

/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn (
/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn (
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release warnings.warn (
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release warnings.warn (

--- running phyghtmap following this gets this error message: Computing hgt/VIEW1/N50E006.hgt hgt file hgt/VIEW1/N50E006.hgt: 3601 x 3601 points, bbox: (6.00000, 50.00000, 7.00000, 51.00000), checking polygon borders /usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/hgt.py:275: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. Computing hgt/VIEW1/N49E006.hgt Traceback (most recent call last):

 File "/usr/local/bin/phyghtmap", line 33, in <module>
   sys.exit(load_entry_point('phyghtmap==2.23', 'console_scripts', 'phyghtmap')())
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 680, in main
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 653, in process
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 552, in processSingleOutput
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 524, in _forkOneSingleOutput
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 438, in processHgtFile
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/main.py", line 390, in writeNodes
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/pbfUtil.py", line 414, in writeNodes
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/pbfUtil.py", line 396, in _makeNodesWays
 File "/usr/local/lib/python3.10/dist-packages/phyghtmap-2.23-py3.10.egg/phyghtmap/pbfUtil.py", line 381, in _makePoints

--Extremecarver (talk) 08:57, 24 February 2023 (UTC)

ValueError: too many values to unpack - Solution

I also had the problem with the error "ValueError: too many values to unpack" in Linux (Mint 21.1 MATE). The solution was to use Agrenott's version mentioned here: User_talk:Panarchos#(Windows)_ValueError:_too_many_values_to_unpack

I can confirm that the problem is due to the changes made in matplotlib. Please make Agrenott's Git Repository the standard source for phyghtmap so that improvement and enhancements can go on!

https://github.com/agrenott/phyghtmap

--ProfMobilux (talk) 07:43, 25 March 2023 (UTC)

New official fork available: pyhgtmap

As it seems phyghtmap is not maintained anymore, I've created a fork and released it on PyPi:

• https://github.com/agrenott/pyhgtmap
• https://pypi.org/project/pyhgtmap/

--Agrenott (talk) 17:35, 16 April 2023 (UTC)