CGImap

From OpenStreetMap Wiki
(Redirected from Cgimap)
Jump to navigation Jump to search


CGImap
CGImap screenshot.png
Author: See list of contributors
License: GNU General Public License v2 (free of charge)
Platform: Linux/Ubuntu 22.04
Version: 0.8.10 (2023-11-05)
Language: English
Source code: https://github.com/zerebubuth/openstreetmap-cgimap
Programming language: C++17

CGImap is a re-implementation of selected performance critical parts of the OSM API 0.6 in C++ (in productive use)



Introduction

CGImap is a C++ implementation of selected performance critical parts of the OSM API v0.6 using the FastCGI interface. It was originally developed to overcome performance limitations present in the openstreetmap-website Rails application, including but not limited to large memory consumption, slow XML parsing, overall slow execution.

As CGImap doesn't manage any database schemas and tables on its own, it can only be deployed alongside the openstreetmap-website application to replace selected API endpoints by their performance optimized counterparts, see Section Production Deployment. CGImap directly interacts with the APIDB, and completely bypasses the openstreetmap-website Rails application.

An important design principle is that for every feature, a Rails port counterpart needs to be implemented first, which doubles as a reference implementation.

Features

Currently, CGImap implements 25 OSM API v0.6 endpoints. This roughly corresponds to 40% of the endpoints implemented on Rails. In production, only 19 endpoints are served by CGImap (Chef repo link to the Apache configuration), all other endpoints are handled by Rails. In particular, single element create/change/delete operations, as well as GPS traces, user data and the map notes API are currently out of scope for CGImap.

  • the "map" API call
  • single node, way and relation fetches
  • multiple node, way and relation fetches
  • the "full" way and relation calls
  • changeset metadata downloads, including discussions
  • single node, way and relation history calls
  • single node, way and relation specific version fetches
  • multiple node, way and relation specific version fetches
  • changeset downloads
  • changeset uploads

(Always check the project GitHub repo for an up-to-date list.)

Supported output formats include OSM XML and OSM JSON. For user authentication Basic auth, OAuth 1.0a or OAuth 2.0 can be used. Note that there are plans to remove both OAuth 1.0a and Basic auth in the future, and focus on OAuth 2.0 instead.[1]

Deviations from Rails openstreetmap-website

  • Diff upload rejects empty keys.
  • Diff upload relies on SQL to create new object version and changeset timestamps, thereby ignores the time reported by the frontend server.
  • Feature Node, way and relation multi-selection calls is not yet available on Rails (Pull request)
  • There is currently no limit for osmChange uploads on the Rails port. CGImap has a configurable default limit of 50MB, based on the uncompressed message.
  • CGImap supports IP-based and user id based rate limiting, with different limits for "normal" users, and moderators. Rails does not have a comparable rate limiting in place.
  • CGImap send an RFC 9110 compliant HTTP Retry-After header, in case too much data has been downloaded. Rails does not send this HTTP header.

Shared code with openstreetmap-website

Since release 0.8.10, CGImap reuses the database function api_rate_limit, which has been introduced in "Add the ability to rate limit edits", and is managed in the Rails repo. Changeset upload rate limiting is off by default, and needs to be enabled by setting the CGIMAP_RATELIMIT_UPLOAD environment variable to true (or the respective --ratelimit-upload command line parameter).

Installation

Installation instructions are available on the project's GitHub repo.

Trivia

Customer quotes

  • We have some hidden magic that makes the website faster for editing. [...] A few years ago we implemented that and you could notice how much quicker the API was.[2]
  • I got it to run as an editable map yesterday but immediately discovered I need cgimap to make it anywhere close to usable.[3]
  • The cgi-map implementation of the diff upload endpoint is blazingly fast[4]
  • CGImap has dramatically improved my experience as a contributor to OpenHistoricalMap. Before, I could barely upload boundary relations without JOSM timing out and retrying, and using iD sometimes felt like reverting to a dialup connection. Now I can upload and download without going out for a walk while I wait.[5]

Faster implementation → less hardware needed:

  • From today the http://OpenStreetMap.org mapping API is now >30% more energy efficient with no reduction in performance. We have removed the need for separate backend API servers.[6]

Notes and references