The Rails Port

From OpenStreetMap Wiki
(Redirected from Rails)
Jump to: navigation, search

The Rails port is the current version of OSM's server code - API, web front end and everything that runs on www.openstreetmap.org. If you have a Unix-like system (Linux, Mac OS X or a BSD) then you can install it on your own machine, so that you can make and test improvements to the code locally, then commit them to the OSM server. (This is not currently trivial for Windows users.)

It is not recommended to set up local copies of The Rails Port for local data editing due to the complexity of later merging the dataset back into the main OSM database. For personal use, it is possible to use an offline editor such as JOSM or Merkaartor to do off-line editing to be saved to a .osm file. This can then be used with other OSM tools, such as Mapnik, and later imported to the main database if required.

Note: In November 2011 The Rails Port was upgraded to run on Rails 3. A large number of changes were made, and the documentation is currently in flux. This will be fully addressed at the Hack Weekend by Gravitystorm and others. If you need assistance before then, please ask on the #osm-dev channel on IRC.

See Rails port/Development for tips on how to contribute your improvements back to the project, and Rails port/UI for style-specific instructions after you've set up the code and want to start hacking on styles.

Contents

Ruby on Rails

Ruby on Rails is a web development framework, written in Ruby. It is intended to make developing database driven web-based applications faster and more productive. You should have some familiarity with Ruby and Model-View-Controller architectures before you start hacking on the OSM Rails port. Here are some resources that might help you:

Installing Rails

Prerequisites/Dependencies

The dependencies aren't particularly stretching for any linux distribution, and are roughly:

Everything else will be installed as RubyGems using Bundler:

Platform-specific instructions

Getting the OpenStreetMap code

Choose a directory to put this code - Rails can run from anywhere on your system. 

 cd <your-base-path-for-source>
 git clone https://github.com/openstreetmap/openstreetmap-website.git

(For commiting changes, see Committing to the rails port)

Rails uses a preconfigured directory structure. Notable directories are:

app/ - The application source code:
app/controllers - The Controllers
app/models - The Models
app/views - The Views - if you just want to work on the look and feel, this is the place to start
db/ - Contains migrations and SQL to create databases (not the actual database)
config/ - Application configuration files
lib/ - OSM Rails specific libraries
log/ - Log files, the directory in svn should be empty
test/ - The Tests (very useful for determining if you have the rails environment and database setup correctly)

Setting up an OSM database

Currently the main API server runs on a PostgreSQL (aka postgres aka pgsql) backend. Before API 0.6 it used a mysql backend.

PostgreSQL setup

Create the openstreetmap user for the postgres database. If you want to use sameuser or other postgres stuff feel free. This method sets a password for the openstreetmap role. 

sudo apt-get install postgresql-contrib libpq-dev
sudo -u postgres -i
createuser openstreetmap -s -P
createdb -E UTF8 -O openstreetmap openstreetmap 
createdb -E UTF8 -O openstreetmap osm_test
createdb -E UTF8 -O openstreetmap osm
psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql

For PostgreSQL >= 9.1, the btree_gist module has changed to be an extension 

psql -d openstreetmap -c "CREATE EXTENSION btree_gist;"
psql -d osm -c "CREATE EXTENSION btree_gist;"

(OS X: see Rails on OS X for the last line)

database.yml configuration

If you want to do any work on the rails port, you'll need to have databases available. Rails uses three databases - one for development, one for testing, and one for production. The configuration for them is in config/database.yml which does not exist by default. So you need to copy config/postgres.example.database.yml to that location.

Once you've copied that file over edit it and insert the username / passwords you provided when you created the databases earlier.

Gems (libraries)

When you have the dependencies and RubyGems installed, you need to install Bundler. Execute as root (sudo on Ubuntu/OS X). 

 # gem install bundler

Now bundler can take care of the rest of the gem: 

 # cd /path/to/rails_port/
 # cp config/example.application.yml config/application.yml
 # cp config/example.database.yml config/database.yml
 # bundle install

Note that if you entered a password for your openstreetmap PostgreSQL user, you need to open script/database.yml uncomment the username and password configuration entries and supply the password you entered.

Database tables

Your database is still empty at this stage. We need to create the tables. Rails handles this by running a series of migrations (see db/migrate), which run quite fast on an empty db! For the development database, simply run 

 rake db:migrate

You don't need to worry about the test database, as "rake test" will take care of it automagically (assuming that you have setup the development database, and created but not run the migrations for the test database). To set up the production database (which you probably won't need), run: 

 env RAILS_ENV=production rake db:migrate

Running the tests

To ensure that everything is set up properly, you should now run: 

rake test

Rake needs the test section of config/database.yml to be setup correctly (use same login and poassword as the develpment section, but a different database)

There may be some errors which does not matter here. To get a chance for a clean test run without errors, install the required functions first as described in the section for pgSQL below.

FIXME: more details here regarding cruise

Populating the database

You can fill the api section of the database with data from planet files using Osmosis - information can be found on the Planet.osm page. This won't add any examples of diaries, gpx traces or fill the history tables. You might not need to do this if you're working on some other bit of the code.

To populate a PostgreSQL database using Osmosis from a planet file, do this: 

osmosis --read-xml-0.6 file="planet.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="openstreetmap" validateSchemaVersion=no

You will need the latest Osmosis to have the right database scheme.

If you intend to write to your database, you will need to reset the auto-increment sequences first (within postgres): 

select setval('acls_id_seq', (select max(id) from acls)); 
select setval('changesets_id_seq', (select max(id) from changesets)); 
select setval('countries_id_seq', (select max(id) from countries)); 
select setval('current_nodes_id_seq', (select max(id) from current_nodes)); 
select setval('current_relations_id_seq', (select max(id) from current_relations)); 
select setval('current_ways_id_seq', (select max(id) from current_ways)); 
select setval('diary_comments_id_seq', (select max(id) from diary_comments)); 
select setval('diary_entries_id_seq', (select max(id) from diary_entries)); 
select setval('friends_id_seq', (select max(id) from friends)); 
select setval('gpx_file_tags_id_seq', (select max(id) from gpx_file_tags)); 
select setval('gpx_files_id_seq', (select max(id) from gpx_files)); 
select setval('messages_id_seq', (select max(id) from messages)); 
select setval('sessions_id_seq', (select max(id) from sessions)); 
select setval('user_tokens_id_seq', (select max(id) from user_tokens)); 
select setval('users_id_seq', (select max(id) from users));

Firing up Rails

The database is now configured and you are ready to roll with rails. Rails comes with its own webserver for testing, called WEBrick. On Unix-like systems as root do: 

# cd /path/to/rails_port/
# rails server

In your favourite web-browser, go to: 

 http://localhost:3000/

You should see the OSM rails port. Congratulations!

Confirming users

If you create a user and you don't want to set up your development box to send E-Mail to a public E-Mail address then you can create the user in the web UI as normal and then confirm it manually through the Rails console: 

$ rails console
>> user = User.find_by_display_name("My New User Name")
=> #[ ... ]
>> user.status = "active"
=> "active"
>> user.save!
=> true
>>

Giving administrator/moderator permissions

To give administrator or moderator permissions do as documented above: 

$ rails console
>> user = User.find_by_display_name("My New User Name")
=> #[ ... ]
>> user.roles.create(:role => "administrator", :granter_id => user.id)
=> #[ ... ]
>> user.roles.create(:role => "moderator", :granter_id => user.id)
=> #[ ... ]
>> user.save!
=> true

Potlatch 2

To use the Potlatch 2 editor you need to register it as an OAuth application. Start by logging into the web site and clicking 'oauth settings' on your user page, then 'Register your application'. When prompted, enter the name of your instance (e.g. "Ruritanian Potlatch 2") and its URL (e.g. "http://openstreetmap.rt/potlatch2"). You also need to check the 'modify the map' box. Everything else can be left with the default values.

Take the "consumer key" which is generated and add it to config/application.yml in your rails tree as the value of the "potlatch2_key" configuration value.

Troubleshooting

Rails has its own log. To inspect the log, do this: 

 tail -f <path-to-osm-source>/sites/rails_port/log/development.log

If you have more problems, please ask on the dev mailing list (dev@openstreetmap.org) or on IRC. Here are some frequently encountered issues.

When running rake db:migrate

   bundle exec rake

When populating the database

Run osmosis on the database "openstreetmap", e.g. use the command 

osmosis --rx file="planet.osm" --wd database="openstreetmap" user="openstreetmap" password="openstreetmap"

Assuming that you are using osmosis with the 0.6 db, you can just add validateSchemaVersion=no to the --write-apidb params.

When booting Rails

When using the site 

 # psql openstreetmap
 select * from users;
 update users set active=1 where id=XX;


Maintaining your installation

If your installation stops working for some reason: 

 rake db:migrate

Getting GPX importing to work

You don't need the GPX import facility to work before using the rest of the Rails port. However, if you want to hack on this bit, too:

If you run into trouble, you can turn on logging by adding this line to lib/daemons/gpx_import_ctl

options[:log_output] = true

Logs will then be written in the log dir.

The high-performance GPX importer

If you are interested in running the same GPX importer as runs on the main server then you should investigate the GPX importer which is present in the git repository at gpx-import but requires you to build C code and configure it. See the settings.sh file in that for help on configuring the higher performance GPX importer

Installing the quadtile functions

This section is not essential and can be skipped by most users. The test suite requires it, however, so if you want a clean test run, you need this.

If you don't install the quadtile functions, you will get three(?) errors when running the test. You can ignore this.

The instructions in db/README for this explain how to do it. In brief:

For PgSQL:

You need to install the *server* extension headers for PostgreSQL, on Ubuntu/Debian that is typically called postgresql-server-dev-8.3 (or 8.2, or whatever version of pgsql you have). 

cd db/functions
make libpgosm.so

 CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 
 AS '/path/to/rails-port/db/functions/libpgosm', 'maptile_for_point'
 LANGUAGE C STRICT;

 CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 
 AS '/path/to/rails-port/db/functions/libpgosm', 'tile_for_point'
 LANGUAGE C STRICT;

The high-performance map call (cgimap)

In addition to the API written in ruby on rails, there is a high-performance implementation of the single API function "map" called cgimap. It is written in C++ as a fastCGI server and is approximately 10x less CPU intense than the rails version. Both versions should give the identical output and can be test statistically with the OsmMapCallValidator

Testing on the osm dev server

For example, after developing a patch for the rails_port, you might want to demonstrate it to others or ask for comments and testing. To do this one can set up an instance of the rails_port on the dev server in ones user directory.

Miscellanea

It is possible to produce diagrams of the rails port using railroad. More information is in /Railroad diagrams.

To generate the HTML documentation of the API/rails code, run the command rake doc:app

To generate test coverage stats, sudo gem install rcov. Then rcov -x gems test/*/*.rb in the rails_port directory.

Some information about testing. The tests are automatically run on commit with the results shown at http://cruise.openstreetmap.org/

To commit your changes, see Rails port/Development

Retrieved from "http://wiki.openstreetmap.org/wiki/The_Rails_Port"
Personal tools
Namespaces
Variants
Actions
site
Toolbox