Nominatim/Installation
These instructions are for the installation of Nominatim V2, which can be found in http://github.com/twain47/Nominatim.git.
For instructions for the older Nominatim V1 that was part of osm2pgsql, see Nominatim/Installation_V1.
Contents
|
Prerequisites
Software
- GCC compiler http://gcc.gnu.org/
- PostgreSQL http://www.postgresql.org/
- Proj4 http://trac.osgeo.org/proj/
- GEOS http://trac.osgeo.org/geos/
- PostGIS http://postgis.refractions.net/
- PHP http://php.net/ (both apache and command line)
- PHP-pgsql
- PEAR::DB http://pear.php.net/package/DB
- protobuf http://code.google.com/p/protobuf/ (for PBF support)
- wget
- git
- osmosis http://wiki.openstreetmap.org/wiki/Osmosis
Ubuntu/Debian
In standard Debian/Ubuntu distributions these should be available as packages.
apt-get install php5-pgsql postgis postgresql php5 php-pear gcc proj-bin libgeos-c1 postgresql-contrib git osmosis apt-get install postgresql-9.1-postgis postgresql-server-dev-9.1 apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev libtool automake libproj-dev
For PBF support you will also need
apt-get install libprotobuf-c0-dev protobuf-c-compiler
Note that the protobuf library that comes with Ubunutu 10.04LTS is too old. You will need to compile and install your own version. More information on the Osm2pgsql page.
In addition, you need to install the pear::DB module:
pear install DB
Note: on some Debian versions, geos requires a second package to be installed. If configure complains about a missing geos/version.h, try
apt-get install libgeos++-dev
Fedora/CentOS
Install these packages from the base and EPEL repositories (CentOS, at least, requires some extra steps to enable EPEL). proj-epsg is required, but broken:
yum install php-pgsql postgis php php-pear php-pear-DB gcc postgresql-server postgresql-contrib postgresql-devel bzip2-devel proj-devel geos-devel git libxml2-devel automake make libtool
Run these commands assuming that you haven't set up postgres:
service postgresql initdb service postgresql start su - postgres createuser -sdr <your username> exit
Run the following command to work around path issues with postgis.sql
ln -s /usr/share/pgsql/contrib/postgis-64.sql /usr/share/pgsql/contrib/postgis-1.5/postgis.sql
There is also a problem with proj-epsg on some platforms, so we manually install it:
wget ftp://ftp.muug.mb.ca/mirror/fedora/epel/6/x86_64/proj-epsg-4.7.0-1.el6.x86_64.rpm rpm -i proj-epsg-4.7.0-1.el6.x86_64.rpm --nodeps
PostgreSQL and PostGIS Version
Currently, PostgreSQL 9.x is recommended, although it should also work with 8.4. 8.3 is no longer supported.
Also, Nominatim makes heavy use of the ST_CONTAINS PostGIS function which seems to be rather slow on PostGIS versions before 1.5, so if you have the option of using PostGIS 1.5 or later, that would be a good idea. Some versions of 1.3 also have stability problems.
Tuning PostgreSQL
You might want to tune your PostgreSQL installation so that the later steps make best use of your hardware. You should tune the following parameters in your postgresql.conf file (e.g. at /etc/postgresql/9.1/main/postgresql.conf in Ubuntu):
- shared_buffers (4GB)
- maintenance_work_mem (16GB/10GB)
- work_mem (50MB)
- effective_cache_size (24GB)
- synchronous_commit = off
- checkpoint_segments = 100
- checkpoint_timeout = 10min
- checkpoint_completion_target = 0.9
The numbers in brackets behind some parameters seem to work fine for 32GB RAM machine. Adjust to your setup. Setting maintenance_work_mem to a relatively high value during the initial import will speed up index creation significantly. However, it should be reduced again afterwards to avoid swapping when autovacuum runs.
For the initial import, you should also set:
- fsync = off
- full_page_writes = off
Don't forget to switch them on again afterwards or you risk database corruption.
Autovacuum must not be switched off because it ensures that the tables are frequently analysed.
PHP configuration
If you get warnings when running the setup.php script:
PHP Warning: file_get_contents(): open_basedir restriction in effect. File(/proc/cpuinfo) is not within the allowed path(s): ...
You need to adjust the open_basedir setting in your PHP configuration (php.ini file). By default this setting may look like this:
open_basedir = /srv/http/:/home/:/tmp/:/usr/share/pear/
Either add reported directories to the list or disable this setting temporarily by adding ";" at the beginning of the line. Don't forget to enable this setting again once you are done with the PHP command line operations.
Hardware
A minimum of 1GB of RAM is required or installation will fail. For a full planet import 32GB or more are recommended.
For a full planet install you will need about 600GB of hard disk space (as of January 2012, take into account that the OSM database is growing fast). SSD disks will help considerably to speed up import and queries but you will need to hack the source code to make use of them.
On poldi the complete initial import required around 2 days. On a 12-core machine with 32GB RAM and standard SATA disks, the initial import (osm2pgsql) takes around 20 hours and the indexing process another 250 hours. Only, 8 parallel threads were used for this setup because I/O speed was the limiting factor. The same machine is able to import the Germany extract in around 4 hours.
First Installation
Getting the source code
You may choose to either use the stable released version (2.0.1) or join the bleeding edge for the latest features and bugs.
Obtaining 2.0.1 (stable release)
Please download and extract the source
wget http://www.nominatim.org/release/Nominatim-2.0.1.tar.bz2 tar xjvf Nominatim-2.0.1.tar.bz2
This version only supports PostgreSQL <= 9.1 and Postgis <= 1.6. If you have newer versions installed, the development version below is needed.
Obtaining the Latest Version
Get the latest source code from Github
git clone --recursive git://github.com/twain47/Nominatim.git
This will pull in the correct version of osm2pgsql as well. If you do not clone recursively, you will need to manually pull the source with git submodule update --init.
Compiling the Required Software
Compile the source:
cd Nominatim ./autogen.sh ./configure make
You can customize Nominatim by creating a local configuration file settings/local.php. Have a look at settings/settings.php for available configuration settings.
Here is an example of a local.php:
<?php
// Paths
@define('CONST_Postgresql_Version', '9.1');
// Website settings
@define('CONST_Website_BaseURL', 'http://mysite/nominatim/');
Download (optional) wikipedia data
Wikipedia can be used as an optional auxiliary data source to help indicate the importance of osm features. Nominatim will work without this information but it will improve the quality of the results if this is installed. This data is available as a binary download.
wget --output-document=data/wikipedia_article.sql.bin http://www.nominatim.org/data/wikipedia_article.sql.bin wget --output-document=data/wikipedia_redirect.sql.bin http://www.nominatim.org/data/wikipedia_redirect.sql.bin
Combined the 2 files are around 1.5GB and add around 30GB to the install size of nominatim. They also increase the install time by an hour or so.
Creating postgres accounts
Creating the importer account
The import needs to be done with a postgres superuser with the same name as the account doing the import. You can create such a postgres superuser account by running:
sudo -u postgres createuser -s <your user name>
This postgres user has to be allowed to do trust authentication (without password - based on Unix username).
Create website user
Create the website user www-data:
createuser -SDR www-data
For the installation process, you must have this user. If you want to run the website under another user, see comment in section Set up the website.
You must not run the import as user www-data.
Nominatim module reading permissions
Some Nominatim Postgres functions are implemented in the nominatim.so C module that was compiled in one of the earlier steps. In order for these functions to be successfully created, PostgreSQL server process has to be able to read the module file. Make sure that directory and file permissions allow the file to be read. For example, if you downloaded and compiled Nominatim in your home directory, you will need to issue the following commands:
chmod +x ~/src chmod +x ~/src/Nominatim chmod +x ~/src/Nominatim/module
Import and index OSM data
First download a Planet File or a planet extract, for example from Geofabrik. Using a file in PBF format is recommended.
Now start the import:
./utils/setup.php --osm-file <your planet file> --all
This will take as little as an hour for a small country extract and as much as 10 days for a full-scale planet import. The import produces a lot of log messages, which you should carefully examine.
Add special phrases
Add country codes and country names to the search index:
./utils/specialphrases.php --countries > specialphrases_countries.sql psql -d nominatim -f specialphrases_countries.sql
If you want to be able to search for special amenities like pubs in Dublin, you need to import special phrases from this wiki like this:
./utils/specialphrases.php --wiki-import > specialphrases.sql psql -d nominatim -f specialphrases.sql
This may be repeated from time to time when there are changes in the wiki. There is no need to repeat it after each update.
If you do not need phrases for all languages, edit utils/specialphrases.php and delete unneeded languages at the beginning of the file.
Set up the website for use with Apache
The following instructions will make Nomiatim available at http://localhost/nominatim.
Create the directory for the website and make sure it is writable by you and readable by apache:
sudo mkdir -m 755 /var/www/nominatim sudo chown <your username> /var/www/nominatim
Populate the website directory with the necessary symlinks:
./utils/setup.php --create-website /var/www/nominatim
You will need to make sure settings/local.php is configured with correct values for CONST_Website_BaseURL.
Also make sure your Apache configuration (/etc/apache2/sites-enabled/000-default in a standard Ubuntu/Debian installation) contains the following settings for the directory:
<Directory "/var/www/nominatim/">
Options FollowSymLinks MultiViews
AddType text/html .php
</Directory>
After making changes in the apache config you need to restart apache.
sudo apache2ctl graceful
Note: The name of the website user is hard-coded into Nominatim. In most Linux distributions, apache will run as www-data, so this will work without any further modifications. If your web server runs under a different name (e.g. in Fedora and CentOS apache runs as user apache), simply alter the name of the www-data user in postgresql after the import has finished, e.g. psql -d nominatim -c 'ALTER USER "www-data" RENAME TO "apache".
Updates
There are many different possibilities to update your Nominatim database. The following section describes how to keep it up-to-date with osmosis. For a list of other methods see the output of ./utils/update.php --help.
Setting up the update process
Next the update needs to be initialised. By default nominatim is configured to update using the global minutely diffs.
If you want a different update source you will need to add some settings to settings/local.php. For example, to use the daily country extracts diffs for Ireland from geofabrik add the following:
@define('CONST_Replication_Url', 'http://download.geofabrik.de/europe/ireland-and-northern-ireland-updates');
@define('CONST_Replication_MaxInterval', '259200'); // Process up to 3 days updates in a run
@define('CONST_Replication_Update_Interval', '86400'); // How often upstream publishes diffs
@define('CONST_Replication_Recheck_Interval', '900'); // How long to sleep if no update found yet
Run the following command to create the osmosis configuration files:
./utils/setup.php --osmosis-init
Enabling hierarchical updates
When a place is updated in the database, all places that contain this place in their address need to be updated as well. These hierarchical updates are disabled by default because they slow down the initial import. Enable them with the following command:
./utils/setup.php --create-functions --enable-diff-updates
Updating Nominatim
The following command will keep your database constantly up to date:
./utils/update.php --import-osmosis-all --no-npi
