Creating a local OSM server under FreeBSD

From OpenStreetMap Wiki
Jump to: navigation, search
Available languages — S-s-s wiki/Creating a local OSM server under FreeBSD
Afrikaans Alemannisch aragonés asturianu azərbaycanca Bahasa Indonesia Bahasa Melayu Bân-lâm-gú Basa Jawa Baso Minangkabau bosanski brezhoneg català čeština dansk Deutsch eesti English español Esperanto estremeñu euskara français Frysk Gaeilge Gàidhlig galego Hausa hrvatski Igbo interlingua Interlingue isiXhosa isiZulu íslenska italiano Kiswahili Kreyòl ayisyen kréyòl gwadloupéyen Kurdî latviešu Lëtzebuergesch lietuvių magyar Malagasy Malti Nederlands Nedersaksies norsk bokmål norsk nynorsk occitan Oromoo oʻzbekcha/ўзбекча Plattdüütsch polski português português do Brasil română shqip slovenčina slovenščina Soomaaliga suomi svenska Tiếng Việt Türkçe Vahcuengh vèneto Wolof Yorùbá Zazaki српски / srpski беларуская български қазақша македонски монгол русский тоҷикӣ українська Ελληνικά Հայերեն ქართული नेपाली मराठी हिन्दी অসমীয়া বাংলা ਪੰਜਾਬੀ ગુજરાતી ଓଡ଼ିଆ தமிழ் తెలుగు ಕನ್ನಡ മലയാളം සිංහල ไทย မြန်မာဘာသာ ລາວ ភាសាខ្មែរ ⵜⴰⵎⴰⵣⵉⵖⵜ አማርኛ 한국어 日本語 中文(简体)‎ 吴语 粵語 中文(繁體)‎ ייִדיש עברית اردو العربية پښتو سنڌي فارسی ދިވެހިބަސް

Description of steps to create a local server OpenStreetMap by Rails_Port v3.* under FreeBSD

Setting up FreeBSD

Tuning the kernel

options         SYSVSHM                 # совместно используемая память SysV
options         SHMMAXPGS=65536
options         SYSVSEM                 # семафоры SysV
options         SEMMNI=40               # максимальное количество наборов семафоров в системе
options         SEMUME=40
options         SEMMNS=240              # макс количество семафоров в системе
options         SEMMNU=120              # макс количество структур undo в системе
options         SYSVMSG                 # межпроцессорное взаимодействие SysV

Install the ports

  • Ruby 1.8.6 or higher
  • RubyGems 1.3.1 or higher
  • Postgres 8.3 or higher
  • ImageMagick

Set in rc.conf

 postgresql_enable="YES"
 postgresql_data="path_to/pgsql/data"
 postgresql_flags="-w -s -m fast"
 postgresql_initdb_flags="--encoding=utf-8 --lc-collate=C"

Set in login.conf:

postgres:\
        :lang=en_US.UTF-8:\
        :setenv=LC_COLLATE=C:\
        :tc=default:

and run

# cap_mkdb login.conf

Get the current OpenStreetMap coge

# git clone git://git.openstreetmap.org/rails.git

Copy and edit the configuration files

# cd /path_to_rails_code
# cp config/example.application.yml config/application.yml
# cp config/example.database.yml config/database.yml 

Work with Postgres

Installing add-ons

# cd /usr/ports/databases/posgresql84-contrib
# make 
# make install

Set in postgresql.conf

 shared_buffers = 128MB
 checkpoint_segments = 20
 maintenance_work_mem = 256MB
 autovacuum = off

Create user and database postgres

# sudo -u pgsql -i
$createuser openstreetmap -s -P
$Enter password for new role:secret
$Enter it again:secret
$createdb -E UTF8 -O openstreetmap openstreetmap 
$createdb -E UTF8 -O openstreetmap osm_test
$createdb -E UTF8 -O openstreetmap osm
$psql -d openstreetmap < /usr/local/share/postgresql/contrib/btree_gist.sql

Setting up access to postgres into pg_hba.conf.

# "local" is for Unix domain socket connections only
local   openstreetmap         openstreetmap                               trust
local   all         all                               ident sameuser
# IPv4 local connections:
host    all         all         127.0.0.1/32          trust
host   openstreetmap         openstreetmap       192.168.0.0/24          trust
host    all         all         192.168.0.0/24          trust

Install the gem-libraries

Install the gem-libraries that are required for code of osm

# gem install bundler
# cd path_to_rails_code/
# bundle install

Note: Some libraries require an additional set of parameters, or installing additional ports.
e.g.:
iconv

# gem install iconv -- --with-iconv-include=/usr/local/include

memached

# cd /usr/ports/databases/memached
# make  -with-sasl
# make install 
# gem install memcached

Other additional gem-libraries

# gem install passenger
Fetching: rake-0.9.2.2.gem (100%)
Fetching: fastthread-1.0.7.gem (100%)
Building native extensions.  This could take a while...
Fetching: daemon_controller-1.0.0.gem (100%)
Fetching: rack-1.4.1.gem (100%)
Fetching: passenger-3.0.13.gem (100%)

The final list of gem-libraries

# gem list --local

*** LOCAL GEMS ***

actionmailer (3.2.2)
actionpack (3.2.2)
activemodel (3.2.2)
activerecord (3.2.2)
activeresource (3.2.2)
activesupport (3.2.2)
addressable (2.2.7)
arel (3.0.2)
builder (3.0.0)
bundler (1.1.5, 1.1.4)
cocaine (0.2.1)
coffee-rails (3.2.2)
coffee-script (2.2.0)
coffee-script-source (1.2.0)
composite_primary_keys (5.0.1)
daemon_controller (1.0.0)
deadlock_retry (1.2.0)
dynamic_form (1.1.4)
erubis (2.7.0)
execjs (1.3.0)
faraday (0.7.6)
fastthread (1.0.7)
hike (1.2.1)
http_accept_language (1.0.2)
httpclient (2.2.4)
i18n (0.6.0)
iconv (0.1)
journey (1.0.3)
jquery-rails (2.0.1)
json (1.6.5)
libv8 (3.3.10.4)
libxml-ruby (2.2.2)
mail (2.4.1)
memcached (1.4.1)
mime-types (1.17.2)
multi_json (1.1.0)
multipart-post (1.1.5)
nokogiri (1.5.0)
oauth (0.4.6, 0.4.5)
oauth-plugin (0.4.0.rc2)
oauth2 (0.5.2)
open_id_authentication (1.1.0)
paperclip (2.7.0)
passenger (3.0.13)
pg (0.13.2)
polyglot (0.3.3)
rack (1.4.1)
rack-cache (1.1)
rack-openid (1.3.1)
rack-ssl (1.3.2)
rack-test (0.6.1)
rails (3.2.2)
rails-i18n (0.5.1)
railties (3.2.2)
rake (0.9.2.2)
rdoc (3.12)
rinku (1.5.1)
ruby-openid (2.1.8)
sanitize (2.0.3)
sass (3.1.15)
sass-rails (3.2.4)
sprockets (2.1.2)
SystemTimer (1.2.3)
therubyracer (0.9.10)
thor (0.14.6)
tilt (1.3.3)
timecop (0.3.5)
treetop (1.4.10)
tzinfo (0.3.31)
uglifier (1.2.3)
validates_email_format_of (1.5.3)

Work with the OpenStreetMap code

Creating and connecting the libpgosm module

# cd /path_to_rails_code/db/functions
# gmake
# sudo -u pgsql -i
$ psql openstreetmap
openstreetmap=# CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 AS '/data1/www/data/map/db/functions/libpgosm.so', 'maptile_for_point' LANGUAGE C STRICT;
openstreetmap=# CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 AS '/data1/www/data/map/db/functions/libpgosm.so', 'tile_for_point' LANGUAGE C STRICT;
$\q

Populating the database structure and test

# sudo -u pgsql -i
$ cd /path_to_rails_code
$ rake db:migrate
$ env RAILS_ENV=production rake db:migrate
$ rake test

Add the roles of administrator and moderator to user

$ /path_to_rails_code/script/rails console
irb(main):001:0> user = User.find_by_display_name("admin")
irb(main):001:0> user.roles.create({:role => "administrator", :granter_id => 1}, :without_protection => true)
irb(main):001:0> user.roles.create({:role => "moderator", :granter_id => 1}, :without_protection => true)
irb(main):004:0> user.save!

Setting up Apache2

Сreate the mod_passenger module

# /usr/local/lib/ruby/gems/1.8/gems/passenger-3.0.13/bin/passenger-install-apache2-module

Сreate a file passenger.conf

apache22/Includes/passenger.conf

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-3.0.7/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-3.0.7
PassengerRuby /usr/local/bin/ruby18

Сreate a file .htaccess

AddHandler fastcgi-script .fcgi
AddHandler cgi-script .cgi
Options +FollowSymLinks +ExecCGI
RewriteEngine On
RewriteRule ^$ index.html [QSA]
RewriteRule ^([^.]+)$ $1.html [QSA]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ public/dispatch.cgi [QSA,L]
ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"

Loading osm-data from a planet file into base

  • Checksum of the planet file.
# gmd5sum -c dump-planet.osm.bz2.md5 dump-planet.osm.bz2
  • Populating the base
# osmosis --read-xml-0.6 file="dump-planet.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="secret" validateSchemaVersion=no

Note: I could not write the downloaded file into the database directly. I had to rewrite it in a file that corresponds to the last date indicated on the website http://planet.openstreetmap.org/day-replicate/.

# bzcat dump-planet.osm.bz2 | osmosis --rri --simc --rx - --ac --wx - | bzip2 > planet-out.osm.bz2
# osmosis --read-xml-0.6 file="planet-out.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="secret" validateSchemaVersion=no

Sources of information

Read more