CGImap
CGImap | |
---|---|
Author: | See list of contributors |
License: | GNU General Public License v2 (free of charge) |
Platform: | Linux/Ubuntu 22.04 |
Version: | 0.8.10 (2023-11-05) |
Language: | English |
Source code: | https://github.com/zerebubuth/openstreetmap-cgimap |
Programming language: | C++17 |
CGImap is a re-implementation of selected performance critical parts of the OSM API 0.6 in C++ (in productive use) |
Introduction
CGImap is a C++ implementation of selected performance critical parts of the OSM API v0.6 using the FastCGI interface. It was originally developed to overcome performance limitations present in the openstreetmap-website Rails application, including but not limited to large memory consumption, slow XML parsing, overall slow execution.
As CGImap doesn't manage any database schemas and tables on its own, it can only be deployed alongside the openstreetmap-website
application to replace selected API endpoints by their performance optimized counterparts, see Section Production Deployment. CGImap directly interacts with the APIDB, and completely bypasses the openstreetmap-website Rails application.
An important design principle is that for every feature, a Rails port counterpart needs to be implemented first, which doubles as a reference implementation.
Features
Currently, CGImap implements 25 OSM API v0.6 endpoints. This roughly corresponds to 40% of the endpoints implemented on Rails. In production, only 19 endpoints are served by CGImap (Chef repo link to the Apache configuration), all other endpoints are handled by Rails. In particular, single element create/change/delete operations, as well as GPS traces, user data and the map notes API are currently out of scope for CGImap.
- the "map" API call
- single node, way and relation fetches
- multiple node, way and relation fetches
- the "full" way and relation calls
- changeset metadata downloads, including discussions
- single node, way and relation history calls
- single node, way and relation specific version fetches
- multiple node, way and relation specific version fetches
- changeset downloads
- changeset uploads
(Always check the project GitHub repo for an up-to-date list.)
Supported output formats include OSM XML and OSM JSON. For user authentication Basic auth, OAuth 1.0a or OAuth 2.0 can be used. Note that there are plans to remove both OAuth 1.0a and Basic auth in the future, and focus on OAuth 2.0 instead.[1]
Deviations from Rails openstreetmap-website
- Diff upload rejects empty keys.
- Diff upload relies on SQL to create new object version and changeset timestamps, thereby ignores the time reported by the frontend server.
- Feature Node, way and relation multi-selection calls is not yet available on Rails (Pull request)
- There is currently no limit for
osmChange
uploads on the Rails port. CGImap has a configurable default limit of 50MB, based on the uncompressed message. - CGImap supports IP-based and user id based rate limiting, with different limits for "normal" users, and moderators. Rails does not have a comparable rate limiting in place.
- CGImap send an RFC 9110 compliant HTTP Retry-After header, in case too much data has been downloaded. Rails does not send this HTTP header.
Since release 0.8.10, CGImap reuses the database function api_rate_limit
, which has been introduced in "Add the ability to rate limit edits", and is managed in the Rails repo. Changeset upload rate limiting is off by default, and needs to be enabled by setting the CGIMAP_RATELIMIT_UPLOAD
environment variable to true (or the respective --ratelimit-upload
command line parameter).
Installation
Installation instructions are available on the project's GitHub repo.
Trivia
Customer quotes
- We have some hidden magic that makes the website faster for editing. [...] A few years ago we implemented that and you could notice how much quicker the API was.[2]
- I got it to run as an editable map yesterday but immediately discovered I need cgimap to make it anywhere close to usable.[3]
- The cgi-map implementation of the diff upload endpoint is blazingly fast[4]
- CGImap has dramatically improved my experience as a contributor to OpenHistoricalMap. Before, I could barely upload boundary relations without JOSM timing out and retrying, and using iD sometimes felt like reverting to a dialup connection. Now I can upload and download without going out for a walk while I wait.[5]
Faster implementation → less hardware needed:
- From today the http://OpenStreetMap.org mapping API is now >30% more energy efficient with no reduction in performance. We have removed the need for separate backend API servers.[6]
Notes and references
- ↑ https://github.com/zerebubuth/openstreetmap-cgimap/issues/286
- ↑ https://2022.stateofthemap.org/sessions/D8XYDN/
- ↑ https://en.osm.town/@mvexel/110373492026784808
- ↑ https://twitter.com/sp8962/status/1120004104061894656
- ↑ https://github.com/OpenHistoricalMap/issues/issues/588
- ↑ https://twitter.com/OSM_Tech/status/1304032683161518090