It's not too complicated, feel free to try it yourself! As a result you will get a specialized map running on you own internet server. You can program this map to display every OpenStreetMap item you want in an extra layer. Furthermore, these items can be made clickable, so extra properties may be shown on the user's request.
Many thanks to Stephan Knauß who developed the first example and licensed it under GPL!
- 1 Prerequisites
- 2 Database
- 3 Choose a Directory
- 4 Tool osm2pgsql
- 5 Tool osmfilter
- 6 Fill the Database
- 7 Supply Web Content
- 8 Test your new Map
- 9 Benchmarks
- 10 Troubleshooting
- 11 Clustering
A weak CPU, e.g. Intel Atom or a virtual Internet server, will suffice, 1 GB RAM are recommended.
We assume that you use Ubuntu >=8.04 as operating system. You must install all programs necessary to run your own web server. Additionally, you will need some other packages. Ensure the installation of all necessary packages by performing these commands:
sudo apt-get install apache2 php5 libapache2-mod-php5 subversion autoconf sudo apt-get install build-essential libxml2-dev libgeos-dev libbz2-dev proj libtool # for Ubuntu <=9.10: sudo apt-get install postgresql-8.3-postgis postgresql-contrib-8.3 postgresql-server-dev-8.3 # for Ubuntu >=10.04: sudo apt-get install postgresql-8.4-postgis postgresql-contrib-8.4 postgresql-server-dev-8.4
The GIS databases must be initialized. Please follow these steps:
sudo -u postgres -i -H createuser -SdR gisuser createdb -E UTF8 -O gisuser gis0 createdb -E UTF8 -O gisuser gis1 createlang plpgsql gis0 createlang plpgsql gis1 # for Ubuntu <=9.10: psql -d gis0 -f /usr/share/postgresql/8.3/contrib/_int.sql psql -d gis1 -f /usr/share/postgresql/8.3/contrib/_int.sql psql -d gis0 -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql psql -d gis1 -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql # for Ubuntu 10.04: psql -d gis0 -f /usr/share/postgresql/8.4/contrib/_int.sql psql -d gis1 -f /usr/share/postgresql/8.4/contrib/_int.sql psql -d gis0 -f /usr/share/postgresql/8.4/contrib/postgis.sql psql -d gis1 -f /usr/share/postgresql/8.4/contrib/postgis.sql # for Ubuntu >=10.10: psql -d gis0 -f /usr/share/postgresql/8.4/contrib/_int.sql psql -d gis1 -f /usr/share/postgresql/8.4/contrib/_int.sql psql -d gis0 -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql psql -d gis1 -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql psql gis0 -c "ALTER TABLE geometry_columns OWNER TO gisuser" psql gis1 -c "ALTER TABLE geometry_columns OWNER TO gisuser" psql gis0 -c "ALTER TABLE spatial_ref_sys OWNER TO gisuser" psql gis1 -c "ALTER TABLE spatial_ref_sys OWNER TO gisuser" exit
To enable easy database login by user gisuser you must change some lines in one of the database configuration files. In case you are running Ubuntu without a graphical interface, please use another editor, e.g. nano instead of gedit.
# for Ubuntu <=9.10: sudo gedit /etc/postgresql/8.3/main/pg_hba.conf # for Ubuntu >=10.04: sudo gedit /etc/postgresql/8.4/main/pg_hba.conf
Near to the bottom of the file you will find these lines:
local all all ident host all all 127.0.0.1/32 md5
Change the words ident and md5 each to trust and close the editor. Now reload the database configuration:
# for Ubuntu <=9.10: sudo /etc/init.d/postgresql-8.3 reload # for Ubuntu >=10.04: sudo /etc/init.d/postgresql-8.4 reload # for Ubuntu 10.10: sudo /etc/init.d/postgresql reload
For a short test, login to the database by using the previously created database user gisuser:
psql gis0 gisuser
Type \d to see a list with the two tables which we have created some steps above (geometry_columns and spatial_ref_sys). Then logout with \q and try the same test with gis1 instead of gis0.
As you have noticed, we use two databases: gis0 and gis1. That is, because we do not want to interrupt web user requests during the daily running process of reconstructing the data base contents. So we use the gis0 database at even numbered days and the gis1 at odd ones. To calculate if a day is even or odd, the UNIX time is retrieved by a "date +%s" command and divided by 86400.
A second database consumes additional memory, of course. But this should not be a serious problem as we are going to condense the OSM planet data to an absolute minimum before we fill the databases.
If you encounter any problems, you may find a solution here: PostGIS.
Choose a Directory
Our example directory for downloading OSM data and generating the data base contents will be "/usr/share/osmgen". So, create this directory and jump into it:
sudo mkdir /usr/share/osmgen cd /usr/share/osmgen
Afterwards, change the access rights of this directory so you have write access with your regular user:
sudo chmod a+rwx .
For security reasons, do not forget to revoke write access as soon as the installation is complete:
sudo chmod -R go-w /usr/share/osmgen # do not run this command now, run it later! sudo chown -R root:root /usr/share/osmgen # do not run this command now, run it later!
We will need the tool osm2pgsql to write OSM data into the database. Although provided by Ubuntu community, it's strongly recommended to build osm2pgsql from source, because we rely on getting the newest version. Please follow these steps:
cd /usr/share/osmgen svn export http://svn.openstreetmap.org/applications/utils/export/osm2pgsql cd osm2pgsql ./autogen.sh ./configure make
Now the databases need to be initialized for Spherical Mercator projection:
cd /usr/share/osmgen psql gis0 gisuser -f osm2pgsql/900913.sql psql gis1 gisuser -f osm2pgsql/900913.sql
If you encounter any problems, you may find a solution here: osm2pgsql.
This tool is used to speed-up the PostgreSQL import process. If we filter out all information we want to use and drop everything else, osm2pgsql will run much faster. On top of this, database queries will be faster too, so the map's user will get the page loaded more fluently.
Downloading the binary of osmfilter (to get the 64 bit version replace "32" by "64"):
cd /usr/share/osmgen wget -O - http://m.m.i24.cc/osmfilter32 >osmfilter chmod ug+x osmfilter
For further details on this tool refer to osmfilter.
Fill the Database
We do not need to store map tiles on our server because we fetch all tiles from http://www.openstreetmap.org/ for performance reasons, but all feature data we want to display on an extra layer must be stored in our database. To do this, we need to develop a script which performs several tasks, step by step. Please do not try to create and run the whole script from scratch. It's better to run-through the process step by step, entering every single command at the command line terminal. That makes it much easier to find errors.
Get the Planet File
It is not recommended to use the file of the entire planet. Please choose the file of an area you are interested in, e.g. Germany. The first task of our script will be to download this file:
cd /usr/share/osmgen wget -O - http://download.geofabrik.de/osm/europe/germany.osm.bz2 2>/dev/null |bunzip2 |gzip -1 >a.osm.gz
Although the file is already available in compressed form, we decided to recompress it, because gzip works a lot faster then bzip2.
Filter the Planet File
./osmfilter a.osm.gz --keep="amenity=restaurant =bar =pub =cafe =fast_food =food_court =nightclub" -o=gis.osm
A few minutes later – dependent of the size of the chosen area – we will get the file gis.osm, containing only that information we need.
Transfer the Data to Postgres Database
The program osm2pgsql will do this job. First you have to ensure that the executable osm2pgsql is available. It should exist in this directory "/usr/share/osmgen/osm2pgsql". Then we need to edit the file "default.style" which should be found in the same directory as the executable.
This default file must be customized to fulfill our needs: all unused tags should be removed by preceding them with a #, and our special tags must be added to this file. As a result, only these lines remain valid, resp. have been added to the file:
# OsmType Tag DataType Flags node,way amenity text nocache,polygon node,way disused text linear node,way name text linear node,way cuisine text linear node,way smokefree text linear node,way smoking text linear node,way smoking_hours text linear node,way smoking:outside text linear
Now we can transfer the OSM data from the file gis.osm into the Postgres database:
osm2pgsql/osm2pgsql -s -C 700 -d gis0 -U gisuser -S osm2pgsql/default.style gis.osm
Did all previous steps work without any errors? Then it's time to pack them into a single script:
cd /usr/share/osmgen gedit gen.sh
That content could be like this:
#!/bin/bash cd /usr/share/osmgen # rotate the log and write a new headline mv gen.log gen.log_temp tail -2000 gen.log_temp>gen.log rm gen.log_temp echo >>gen.log echo Starting update script >>gen.log date >>gen.log # get the latest .osm file # download section - start rm a.osm.gz echo Downloading .osm file >>gen.log wget -O - http://download.geofabrik.de/osm/europe/germany.osm.bz2 2>/dev/null |bunzip2 |gzip -1 >a.osm.gz # download section - end ls -lL a.osm.gz >>gen.log if [ "0"$(stat -L -c%s a.osm.gz) -lt 900000 ]; then echo "Downloaded file too small (implausible)" >>gen.log date >>gen.log exit fi date >>gen.log # filter the .osm file ./osmfilter a.osm.gz --keep="amenity=restaurant =bar =pub =cafe =fast_food =food_court =nightclub" -o=gis.osm echo File has been filtered. >>gen.log date >>gen.log # generate database contents d=$(date +%s) echo Unix time: $d >>gen.log d=$(expr $d + 39600 + 300) # 11*3600 means: at about 11:00 UT; # i.e., this script should be started at 11:00 UT (or later) d=$(expr $d / 86400 % 2) #d=$(expr 1 - $d) echo Determined database name: gis$d >>gen.log echo Generating data base tables >>gen.log ./osm2pgsql -s -C 700 -d gis$d -U gisuser -S gastro.style gis.osm 2>/dev/null >/dev/null date >>gen.log echo >>gen.log
The typical process duration time for this script is 7 minutes on a slow machine (virtual web server with 1 GB RAM assigned):
- 10 minute for getting the planet file (Germany),
- 5 minutes for unpacking/filtering, and
- 1 minute for writing the result into the database.
Weekly Map Data Update
To get the map data updated automatically every week, an additional Cron entry will be mandatory. For example, execute this command to start the update process every Sunday at 3 p.m.:
sudo echo -e "0 15 * * 7 root /usr/share/osmgen/gen.sh 2>&1>/dev/null \n" > /etc/cron.d/osmgen
Daily Map Data Update
Instead of downloading the whole regional .osm file, you can also use the mechanism for a daily update. After you have followed the description in that page, create a symbolic link to the automatically updated .osm file (insert the right path):
cd /usr/share/osmgen ln -fs /insert_here_the_actual_path_to_osmupdate_directory/a.osm.gz a.osm.gz
If you use the shell script from above, you will no longer need the script's download part. Hence, remove the 3 lines which are included by the # download section markings.
Supply Web Content
The web server Apache will expect our web content at "/var/www". So, change to that directory and install all necessary files:
16pxb.png 16pxg.png 16pxr.png 16pxy.png ajax-loader.gif b.png cb.png cg.png cr.png cui_afghan.png ... cui_vietnamese.png cy.png features.php g.png index.html r.png y.png
The most important files can be found here:
Note that you will need write access rights in the www directory. If these rights have not been set automatically during Apache installation, you may want to do this by hand:
sudo chown root:www-data /var/www sudo chmod g+rwx /var/www sudo usermod -aG www-data $USER
To get these changes work, please relogin.
Because we are going to use some other libraries, download copies of their files and place them into your www directory. Afterwards delete all library files we do not need:
cd /var/www wget http://code.jquery.com/jquery.min.js wget http://www.openstreetmap.org/openlayers/OpenStreetMap.js wget http://openlayers.org/download/OpenLayers-2.9.zip unzip OpenLayers-2.9.zip mv OpenLayers-2.9 ol cd ol rm -r apidoc_config build doc doc_config examples lib tests tools
Test your new Map
Open a web browser and try to access your new website. If your browser is installed on the same computer as the Apache server, type localhost as URL.
(Please add comments.)
(chapter needs to be written...)
Stephan developed a nice server sided clustering system