Batch Upload of GPX (various tools)

From OpenStreetMap Wiki
Jump to: navigation, search
Available languages — Batch_Upload
Afrikaans Alemannisch aragonés asturianu azərbaycanca Bahasa Indonesia Bahasa Melayu Bân-lâm-gú Basa Jawa Baso Minangkabau bosanski brezhoneg català čeština dansk Deutsch eesti English español Esperanto estremeñu euskara français Frysk Gaeilge Gàidhlig galego Hausa hrvatski Igbo interlingua Interlingue isiXhosa isiZulu íslenska italiano Kiswahili Kreyòl ayisyen kréyòl gwadloupéyen Kurdî latviešu Lëtzebuergesch lietuvių magyar Malagasy Malti Nederlands Nedersaksies norsk bokmål norsk nynorsk occitan Oromoo oʻzbekcha/ўзбекча Plattdüütsch polski português português do Brasil română shqip slovenčina slovenščina Soomaaliga suomi svenska Tiếng Việt Türkçe Vahcuengh vèneto Wolof Yorùbá Zazaki српски / srpski беларуская български қазақша македонски монгол русский тоҷикӣ українська Ελληνικά Հայերեն ქართული नेपाली मराठी हिन्दी অসমীয়া বাংলা ਪੰਜਾਬੀ ગુજરાતી ଓଡ଼ିଆ தமிழ் తెలుగు ಕನ್ನಡ മലയാളം සිංහල ไทย မြန်မာဘာသာ ລາວ ភាសាខ្មែរ ⵜⴰⵎⴰⵣⵉⵖⵜ አማርኛ 한국어 日本語 中文(简体)‎ 吴语 粵語 中文(繁體)‎ ייִדיש עברית اردو العربية پښتو سنڌي فارسی ދިވެހިބަސް

Main idea is to upload GPX files directly from scripts, without having to use the web interface. Note that this isn't the same thing as trying to upload OpenStreetMap data itself; for that see Editing or Imports.

Authenticaton

The scripts here use HTTP Basic Auth to function. If you are writing a new uploader you should consider using OAuth instead. See the examples page for a ruby script that uploads GPX traces using OAuth.

Tools

Tar or Zip

The simplest way is just to zip up your gpx files and submit a single zip (or tar or tar.gz) file using the normal upload form. The only drawback with this is that you can only attach one description to the whole file set, and it will appear as a single trace in the upload log.

Perl

You can upload a GPX to the API like this:

use HTTP::Request::Common;
use LWP::UserAgent;

$ua=LWP::UserAgent->new;
$ua->credentials('www.openstreetmap.org:80','Web Password',$yourusername, $yourpassword);

$response=$ua->request(POST 'http://www.openstreetmap.org/api/0.6/gpx/create',
	Content_Type => 'form-data',
	Content	     => [ file       =>[$filename],
			  description=>$description,
			  tags       =>$tags,
			  visibility =>"public" ] );

if ($response->code==200) {
	# yay, success
} else {
	# boo, failure
}

A fully working batch uploader, then, might look like this:

#!/usr/bin/perl -w

use HTTP::Request::Common;
use LWP::UserAgent;

$yourusername='';	# put your OSM username between the quotes
$yourpassword='';	# put your OSM password between the quotes

foreach $filename (@ARGV) {

	$ua=LWP::UserAgent->new;
	$ua->credentials('www.openstreetmap.org:80','Web Password',$yourusername, $yourpassword);

	print "Uploading $filename\n";
	print "Description: "; $description=<STDIN>; chomp $description;
	print "Tags (comma-separated): "; $tags=<STDIN>; chomp $tags;

	$response=$ua->request(POST 'http://www.openstreetmap.org/api/0.6/gpx/create',
			Content_Type => 'form-data',
			Content	     => [ file       =>[$filename],
					  description=>$description,
					  tags       =>$tags,
					  visibility =>"public" ] );

	if ($response->code==200) {
		# yay, success
		print "uploaded successfully\n";
	} else {
		# boo, failure
		print "couldn't upload $filename\n";
	}
}
Save this as upload.pl in the same directory as your .gpx files. Then go to the command line and type
perl upload.pl *.gpx
to upload every file. You'll be prompted for a description and tags for each one. There is a version that prompts with a graphical user interface (using Zenity) here: http://gist.github.com/444526

Java

It is only one java class (needs compiling) and it allows to pass all uploaded files to other command line apps (using xargs).

Feedback is welcome: Christof Dallermassl

Usage

java GpxUpload <description> <tags> <files*>

Osm username and password can be defined as system properties by -Dusername=<username> and -Dpassword=<password> or if not given, josm's preference file is read. Any messages are printed to stderror, only the filename that was sent successfully is printed to stdout, so you may use the output of this program in a pipe for other calls.

Examples

java GpxUpload "taking a ride in Graz, Austria" "graz austria" gpxfile.gpx
java GpxUpload "taking a ride in Graz, Austria" "graz austria" gpxfiles*.gpx | xargs -i mv '{}' /home/cdaller/targetdir

Download

http://svn.openstreetmap.org/applications/utils/gps-tracks/jgpxupload/

C/libcurl

The following source code works with glib+libcurl.

void osm_traces_upload_file(const char *user, const char *password, const char *file, const char *filename,
				const char *description, const char *tags, gboolean public)
{
  CURL *curl;
  CURLcode res;
  char curl_error_buffer[CURL_ERROR_SIZE];
  struct curl_slist *headers = NULL;
  struct curl_httppost *post=NULL;
  struct curl_httppost *last=NULL;
  gchar *visibility_string;
  char *base_url = "http://www.openstreetmap.org/api/0.6/gpx/create";
  gchar *user_pass = get_login();
  /* Init CURL */
  curl = curl_easy_init();
  /* Filling the form */
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "description",
               CURLFORM_COPYCONTENTS, description, CURLFORM_END);
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "tags",
               CURLFORM_COPYCONTENTS, tags, CURLFORM_END);
  if (public) visibility_string = "public";
  else        visibility_string = "private";
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "visibility",
               CURLFORM_COPYCONTENTS, visibility_string, CURLFORM_END);
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "file",
               CURLFORM_FILE, file,
               CURLFORM_FILENAME, filename,
               CURLFORM_CONTENTTYPE, "text/xml", CURLFORM_END);
  /* Prepare request */
  /* As explained in http://wiki.openstreetmap.org/index.php/User:LA2 */
  /* Expect: header seems to produce incompatibilites between curl and httpd */
  headers = curl_slist_append(headers, "Expect: ");
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
  curl_easy_setopt(curl, CURLOPT_HTTPPOST, post);
  curl_easy_setopt(curl, CURLOPT_URL, base_url);
  curl_easy_setopt(curl, CURLOPT_USERPWD, user_pass);
  curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
  curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, &curl_error_buffer);
  /* Execute request */
  res = curl_easy_perform(curl);
  if (res == CURLE_OK)
  {
    long code;
    res = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &code);
    if (res == CURLE_OK)
    {
      g_debug("received valid curl response: %ld", code);
      if (code != 200) g_warning("failed to upload data: HTTP response is %ld", code);
    }
    else g_error("curl_easy_getinfo failed: %d", res);
  else g_warning("curl request failed: %s", curl_error_buffer);
  /* Memory */
  g_free(user_pass); user_pass = NULL;
  curl_formfree(post);
  curl_easy_cleanup(curl); 
}

CURL

This method will fail if tags or description contains non-ascii chars.

curl -u user:password -H "Expect: " -F "file=@"file.gpx -F description=description \
   -F tags=tags -F visibility=public http://www.openstreetmap.org/api/0.6/gpx/create

file.gpx is your filename, description, tags, and visibility, as well as user and password must be changed.

More Information

See the API documentation for more details on the api methods available.