Talk:Servers

From OpenStreetMap

Jump to: navigation, search

What about using PostgreSQL as the database server? It greatly outperforms MySQL when subjected to lots of concurrent connections which OSM is likely to experience.

Regarding SATA/SAS v.s. SCSI I respectfully recommend you revise your opinion, most reviews now recommend the use of SATA/SAS for database applications for a number of reasons. Modern and properly selected SATA/SAS RAID solutions easilly outperform equally expensive SCSI solutions in speed, flexibility, scalability and reliability (through the use of more harddisks). --Lambertus 18:32, 3 March 2007 (UTC)

PostgreSQL has been looked at for this specific application and found to offer no improvement; anyone who can prove PosgreSQL superiority in a scenario similar to our production requirement is very welcome to step forward with numbers. (That's what I gathered from reading the mailing lists; "Postgres is better" comes up once every six weeks but is never really proved.) --Frederik Ramm 12:46, 9 March 2007 (UTC)
no real test was ever done. the proposition is dispelled without even trying
Frederik: Please take a look at my link regarding MySQL v.s. PostgreSQL, that article provides hard numbers from a thorough test where it shows that PostgreSQL - under high concurrent loads as OSM is likely to experience - leaves MySQL in the dust. I withdraw my comment regarding OpenGIS after reading more on the mailing lists though. --Lambertus 18:37, 9 March 2007 (UTC)
Another fact about the chosen MySQL database engine MyISAM is given by the developers of MySQL itself (source): 
 MySQL gives database developers the choice of which approach to use. 
 If you don't need foreign keys and want to avoid the overhead associated with enforcing referential integrity, you can choose another storage engine instead, such as MyISAM. 
 (For example, the MyISAM storage engine offers very fast performance for applications that perform only INSERT and SELECT  operations. ....)
The above quote shows clearly that choosing the MyISAM database engine for a project like OSM where high amounts of update/delete (along insert and select) queries happen concurrently AND where referential integrity is required is not very sane. This project at least needs a database engine that allows for row-level locking which would mean that OSM has to choose between MySQL's innoDB or PostgreSQL, where the link in my previous comment clearly shows that PostgreSQL wins hands down. --Lambertus 11:34, 21 March 2007 (UTC)

Tyan motherboards are recognized as being very good. the S2927 while being socket F has ample room for expansion and can handle up to 32G of ram. it not only supports Gnu/Linux, but will soon be supported by LinuxBios

I agree the Tyan motherboards are good, and durable. There is a balancing act to draw. For a server which needs to be installed for several years and not touched, the arguments are compelling and the prices justifiable. An opteron 2218 for the Tyan motherboard will cost £368 vs a similar vanilla dual core costing £141.36. A nb2600b will cost £226 vs a single socket at around £60. In other words, the expandability option for the extra CPU socket and 4 DIMM sockets raises the cost by £422. I guess the better solution is to try to hit the sweet spot in computing/memory costs then split the heaviest table workload between more than one machine. That is the current proposal, re-assigning the current DB to GPX work and the new DB concentrate on OSM map data, so we can continue using the cheapest vanilla components. NickH 20:47, 9 March 2007 (UTC)

  • why pay that much for a 2218 when you can get 2 2214 for the same price, or 2 2216 for just a tad more ? the extra cores would run your database server much faster, allowing for 2 more reading threads (or something) at the same time

The same argument could be used for SATA vs SCSI, and in general, that is an argument i'd accept. However, I have found I can get lightly used very high performance SCSI drives at good prices, and used SCSI raid controllers, again, at compelling prices. This completely changes the economics of the SCSI vs SATA debate, before we get to the issues of relative performance. I will take another look at the SCSI vs SATA debate and read the following reviews this evening, but looking at the price of SATA adapters, I expect to pay more for an 8 way SATA controller than a good SCSI raid controller plus 4 high performance hard drives. My current opinion is that I would need 8 SATA drives to give the performance of 4 high spec SCSI devices.NickH 20:47, 9 March 2007 (UTC)

I'll reiterate my recommandation to using areca Sata controllers. see this article and this for more information. Oh, and should I add to the mix the various papers that came out recently on Disks reliability (or rather lack of it) that is similar for all technologies.

Not arguing against secondhand hardware, I would like to compare prices (sorry: Dutch) when buying new I/O subsystem hardware. I introduce four options (2 SATA, 2 SCSI) and compare the prices:
  • A:
1x Areca 1120 SATA RAID controller with 256MB cache supporting 8 disks: EUR 490
8x Western Digital Raptor 73GB (10k rpm): EUR 137, total EUR 1096
Total price: EUR 1586
  • B:
1x Areca 1160 SATA RAID controller with 256MB cache supporting 16 disks: EUR 941
16x Seagate Barracuda 7200.9 80GB (7200 rpm): EUR 42, total EUR 672
Total price: EUR 1613
  • C:
1x Adaptec ASR2230SLP SCSI RAID controller with 256MB cache supporting 2 UW320 channels: EUR 587
8x Seagate Cheetah 10k7 73GB (10k rpm): EUR 150, total EUR 1200
Total price: EUR 1787
  • D:
1x Adaptec ASR2230SLP SCSI RAID controller with 256MB cache supporting 2 UW320 channels: EUR 587
4x Seagate Cheetah 15k4 73GB (15k rpm): EUR 267, total EUR 1086
Total price: EUR 1655
While the 4x 15k rpm disks of option D may seem a good solution (low latency, moderate price), putting them in RAID-5 will cut back the performance considerably according to: PostgreSQL performance tuning. As the performance of databases greatly depends on the number of heads, the seemingly slow option B allows for 6x the amount of heads at the same price (not to mention the amount of available storage). Option A is about 12% cheaper compared to option C but is also 10 to 20% slower in database environments. Option (or using even less disks) should perform best when using e.g. two RAID-1 sets (one for OS+transaction logging, one for data). --Lambertus 16:05, 21 March 2007 (UTC)

sxpert said: 

   £10 bank fees on 50€ ??? wtf ?

Yes, my bank obviously charge a fixed fee for certain things, not a percentage. Swings and roundabouts. The 90E payment didn't attract any fee. In this case, a forex dealing fee. If the exchange was made at the sending bank, no fee would have been applied by my bank, but the sending bank may have levied a larger fee.NickH

  • there would be no change charge if you people would accept switching to the euro :D

That is waaaay off topic.

Is it possible to split the database server into machines serving different areas? The API could look-up latitude and longitude quickly and then send the data to the appropriate machine. Multiple machines tend to cheaper than one big machine. Is this more complicated than it sounds?

Server Naming Theme Discussion and Vote

Voting has now closed

Results: 

Cartographers : 12
Explorers : 11
Geographers : 9 
Topologists : 8
Famous Fictional Cats : 10
Fictional dragons ("here be dragons") : 18
Popular "amenity" tag values : 6 
GPS / GIS formats : 2
National Mapping Agencies : 2
Word "map" in different languages : 9
Capital cities : 10


Our servers need new names. Its your turn to come up with an idea for a theme. We need to be able to pick nice simple easy to remember names from a common theme.

Retrieved from "http://wiki.openstreetmap.org/index.php/Talk:Servers"
Personal tools
recent changes