Batch Upload of GPX (various tools)

Main idea is to upload GPX files directly from scripts, without having to use the web interface. Note that this isn't the same thing as trying to upload OpenStreetMap data itself; for that see Editing or Imports.

Authenticaton

Most of the scripts here use HTTP Basic Auth to function. If you are writing a new uploader you should consider using OAuth instead. See the examples page for a ruby script that uploads GPX traces or a Python script below using OAuth.

Tools

Tar or Zip

The simplest way is just to zip up your gpx files and submit a single zip (or tar or tar.gz) file using the normal upload form. The only drawback with this is that you can only attach one description to the whole file set, and it will appear as a single trace in the upload log.

Perl

You can upload a GPX file to the API like this:

use HTTP::Request::Common;
use LWP::UserAgent;

# Log in
$ua=LWP::UserAgent->new;
$ua->credentials('www.openstreetmap.org:443','Web Password',$yourusername, $yourpassword);

$response=$ua->request(POST 'https://www.openstreetmap.org/api/0.6/gpx/create',
	Content_Type => 'form-data',
	Content	     => [ file       =>[$filename],
			  description=>$description,
			  tags       =>$tags,
			  visibility =>"identifiable" ] );

if ($response->code==200) {
	# yay, success
} else {
	# boo, failure
}

A fully working batch uploader, then, might look like this:

#!/usr/bin/perl -w

use HTTP::Request::Common;
use LWP::UserAgent;

$yourusername='';	# put your OSM username between the quotes
$yourpassword='';	# put your OSM password between the quotes

# Log in
$ua=LWP::UserAgent->new;
$ua->credentials('www.openstreetmap.org:443', 'Web Password', $yourusername, $yourpassword);

foreach $filename (@ARGV) {
	print "Uploading $filename\n";
	print "Description: "; $description=<STDIN>; chomp $description;
	print "Tags (comma-separated): "; $tags=<STDIN>; chomp $tags;

	$response=$ua->request(POST 'https://www.openstreetmap.org/api/0.6/gpx/create',
			Content_Type => 'form-data',
			Content	     => [ file       =>[$filename],
					  description=>$description,
					  tags       =>$tags,
					  visibility =>"identifiable" ] );

	if ($response->code==200) {
		# yay, success
		print "uploaded successfully\n";
	} else {
		# boo, failure
		print "couldn't upload $filename: " . $response->status_line . "\n";
	}
}

Save this as upload.pl in the same directory as your .gpx files. Then go to the command line and type

perl upload.pl *.gpx

to upload every file. You'll be prompted for a description and tags for each one. There is a version that prompts with a graphical user interface (using Zenity) here: http://gist.github.com/444526

Python

#!/usr/bin/env python3
from getpass import getpass
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient

print('''
Visit https://www.openstreetmap.org/oauth2/applications/new

Fill in the form:
 Name: arbitrary
 Redirect URIs: https://localhost
 [x] Confidential application
 Permissions:
 [x] Upload GPS traces

Paste Client ID and secret below.

''')

client_id = input('Client ID: ')
client_secret = getpass('Client secret (not echoed): ')

redirect_uri = 'https://localhost'
oauth = OAuth2Session(client_id, redirect_uri=redirect_uri, scope='write_gpx')
authorization_url, state = oauth.authorization_url('https://www.openstreetmap.org/oauth2/authorize')
print(f'''
Open this URL in browser: {authorization_url}
Authorize the application and paste the resulting URL below (ignore connection error in the browser window)
''')
authorization_response = input()

data = oauth.fetch_token('https://www.openstreetmap.org/oauth2/token', authorization_response=authorization_response, client_secret=client_secret)

print(f'''
Use curl to upload GPX:

curl -H "Authorization: Bearer {data['access_token']}" -F file="@$1" -F description=Foo -F tags=Bar -F visibility=identifiable https://www.openstreetmap.org/api/0.6/gpx/create
''')

Save the resulting curl command to a file and execute it providing a gpx file to upload as an argument.

Java

It is only one java class (needs compiling) and it allows to pass all uploaded files to other command line apps (using xargs).

Feedback is welcome: Christof Dallermassl

Usage

java GpxUpload <description> <tags> <files*>

Osm username and password can be defined as system properties by -Dusername=<username> and -Dpassword=<password> or if not given, josm's preference file is read.

Any messages are printed to stderror, only the filename that was sent successfully is printed to stdout, so you may use the output of this program in a pipe for other calls.

Examples

java GpxUpload "taking a ride in Graz, Austria" "graz austria" gpxfile.gpx

java GpxUpload "taking a ride in Graz, Austria" "graz austria" gpxfiles*.gpx | xargs -i mv '{}' /home/cdaller/targetdir

Download

http://svn.openstreetmap.org/applications/utils/gps-tracks/jgpxupload/

C/libcurl

The following source code works with glib+libcurl.

void osm_traces_upload_file(const char *user, const char *password, const char *file, const char *filename,
				const char *description, const char *tags, gboolean public)
{
  CURL *curl;
  CURLcode res;
  char curl_error_buffer[CURL_ERROR_SIZE];
  struct curl_slist *headers = NULL;
  struct curl_httppost *post=NULL;
  struct curl_httppost *last=NULL;
  gchar *visibility_string;
  char *base_url = "https://www.openstreetmap.org/api/0.6/gpx/create";
  gchar *user_pass = get_login();
  /* Init CURL */
  curl = curl_easy_init();
  /* Filling the form */
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "description",
               CURLFORM_COPYCONTENTS, description, CURLFORM_END);
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "tags",
               CURLFORM_COPYCONTENTS, tags, CURLFORM_END);
  if (public) visibility_string = "identifiable"; /* Use new recommendations */
  else        visibility_string = "trackable";
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "visibility",
               CURLFORM_COPYCONTENTS, visibility_string, CURLFORM_END);
  curl_formadd(&post, &last,
               CURLFORM_COPYNAME, "file",
               CURLFORM_FILE, file,
               CURLFORM_FILENAME, filename,
               CURLFORM_CONTENTTYPE, "text/xml", CURLFORM_END);
  /* Prepare request */
  /* As explained in http://wiki.openstreetmap.org/index.php/User:LA2 */
  /* Expect: header seems to produce incompatibilites between curl and httpd */
  headers = curl_slist_append(headers, "Expect: ");
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
  curl_easy_setopt(curl, CURLOPT_HTTPPOST, post);
  curl_easy_setopt(curl, CURLOPT_URL, base_url);
  curl_easy_setopt(curl, CURLOPT_USERPWD, user_pass);
  curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
  curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, &curl_error_buffer);
  /* Execute request */
  res = curl_easy_perform(curl);
  if (res == CURLE_OK)
  {
    long code;
    res = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &code);
    if (res == CURLE_OK)
    {
      g_debug("received valid curl response: %ld", code);
      if (code != 200) g_warning("failed to upload data: HTTP response is %ld", code);
    }
    else g_error("curl_easy_getinfo failed: %d", res);
  else g_warning("curl request failed: %s", curl_error_buffer);
  /* Memory */
  g_free(user_pass); user_pass = NULL;
  curl_formfree(post);
  curl_easy_cleanup(curl); 
}

CURL

If your copy of curl is running under a non-UTF-8 environment, this method will fail if tags or description contains non-ascii chars as they would form invalid UTF-8. Making sure that you are passing the values in UTF-8 may help with such environments: -F description="$(iconv -t UTF-8 <<< "weird café")".

curl -u 'user':'password' -H "Expect: " -F file=@"file.gpx" -F description="description here" \
   -F tags="tags here" -F visibility=identifiable https://www.openstreetmap.org/api/0.6/gpx/create

file.gpx is your filename, description, tags, and visibility, as well as user and password must be changed.

More Information

See the API documentation for more details on the api methods available.