Talk:Servers

From OpenStreetMap Wiki
Jump to navigation Jump to search

Discuss Servers page here:


MySQL vs PostgreSQL

What about using PostgreSQL as the database server? It greatly outperforms MySQL when subjected to lots of concurrent connections which OSM is likely to experience.--Lambertus 18:32, 3 March 2007 (UTC)

PostgreSQL has been looked at for this specific application and found to offer no improvement; anyone who can prove PosgreSQL superiority in a scenario similar to our production requirement is very welcome to step forward with numbers. (That's what I gathered from reading the mailing lists; "Postgres is better" comes up once every six weeks but is never really proved.) --Frederik Ramm 12:46, 9 March 2007 (UTC)
no real test was ever done. the proposition is dispelled without even trying -- User:Sxpert 14:09, 9 March 2007
Frederik: Please take a look at my link regarding MySQL v.s. PostgreSQL, that article provides hard numbers from a thorough test where it shows that PostgreSQL - under high concurrent loads as OSM is likely to experience - leaves MySQL in the dust. I withdraw my comment regarding OpenGIS after reading more on the mailing lists though. --Lambertus 18:37, 9 March 2007 (UTC)
Another fact about the chosen MySQL database engine MyISAM is given by the developers of MySQL itself (source):
MySQL gives database developers the choice of which approach to use.
If you don't need foreign keys and want to avoid the overhead associated with enforcing referential integrity, you can choose another storage engine instead, such as MyISAM.
(For example, the MyISAM storage engine offers very fast performance for applications that perform only INSERT and SELECT operations. ....)
The above quote shows clearly that choosing the MyISAM database engine for a project like OSM where high amounts of update/delete (along insert and select) queries happen concurrently AND where referential integrity is required is not very sane. This project at least needs a database engine that allows for row-level locking which would mean that OSM has to choose between MySQL's innoDB or PostgreSQL, where the link in my previous comment clearly shows that PostgreSQL wins hands down. --Lambertus 11:34, 21 March 2007 (UTC)

Tyan motherboards

Tyan motherboards are recognized as being very good. the S2927 while being socket F has ample room for expansion and can handle up to 32G of ram. it not only supports Gnu/Linux, but will soon be supported by LinuxBios User:Sxpert 10:02, 9 March 2007

I agree the Tyan motherboards are good, and durable. There is a balancing act to draw. For a server which needs to be installed for several years and not touched, the arguments are compelling and the prices justifiable. An opteron 2218 for the Tyan motherboard will cost £368 vs a similar vanilla dual core costing £141.36. A nb2600b will cost £226 vs a single socket at around £60. In other words, the expandability option for the extra CPU socket and 4 DIMM sockets raises the cost by £422. I guess the better solution is to try to hit the sweet spot in computing/memory costs then split the heaviest table workload between more than one machine. That is the current proposal, re-assigning the current DB to GPX work and the new DB concentrate on OSM map data, so we can continue using the cheapest vanilla components. NickH 20:47, 9 March 2007 (UTC)
why pay that much for a 2218 when you can get 2 2214 for the same price, or 2 2216 for just a tad more ? the extra cores would run your database server much faster, allowing for 2 more reading threads (or something) at the same time - User:Sxpert 21:43, 9 March 2007

SATA/SAS v.s. SCSI

Regarding SATA/SAS v.s. SCSI I respectfully recommend you revise your opinion, most reviews now recommend the use of SATA/SAS for database applications for a number of reasons. Modern and properly selected SATA/SAS RAID solutions easilly outperform equally expensive SCSI solutions in speed, flexibility, scalability and reliability (through the use of more harddisks). --Lambertus 18:32, 3 March 2007 (UTC)

I'll reiterate my recommandation to using areca Sata controllers. see this article and this for more information. Oh, and should I add to the mix the various papers that came out recently on Disks reliability (or rather lack of it) that is similar for all technologies. User:Sxpert 07:26, 9 March 2007
The same argument (['balancing act to draw' above]) could be used for SATA vs SCSI, and in general, that is an argument i'd accept. However, I have found I can get lightly used very high performance SCSI drives at good prices, and used SCSI raid controllers, again, at compelling prices. This completely changes the economics of the SCSI vs SATA debate, before we get to the issues of relative performance. I will take another look at the SCSI vs SATA debate and read the following reviews this evening, but looking at the price of SATA adapters, I expect to pay more for an 8 way SATA controller than a good SCSI raid controller plus 4 high performance hard drives. My current opinion is that I would need 8 SATA drives to give the performance of 4 high spec SCSI devices.NickH 20:47, 9 March 2007 (UTC)


Prices I/O subsystem

Not arguing against secondhand hardware, I would like to compare prices (sorry: Dutch) when buying new I/O subsystem hardware. I introduce four options (2 SATA, 2 SCSI) and compare the prices:
  • A:
1x Areca 1120 SATA RAID controller with 256MB cache supporting 8 disks: EUR 490
8x Western Digital Raptor 73GB (10k rpm): EUR 137, total EUR 1096
Total price: EUR 1586
  • B:
1x Areca 1160 SATA RAID controller with 256MB cache supporting 16 disks: EUR 941
16x Seagate Barracuda 7200.9 80GB (7200 rpm): EUR 42, total EUR 672
Total price: EUR 1613
  • C:
1x Adaptec ASR2230SLP SCSI RAID controller with 256MB cache supporting 2 UW320 channels: EUR 587
8x Seagate Cheetah 10k7 73GB (10k rpm): EUR 150, total EUR 1200
Total price: EUR 1787
  • D:
1x Adaptec ASR2230SLP SCSI RAID controller with 256MB cache supporting 2 UW320 channels: EUR 587
4x Seagate Cheetah 15k4 73GB (15k rpm): EUR 267, total EUR 1086
Total price: EUR 1655
While the 4x 15k rpm disks of option D may seem a good solution (low latency, moderate price), putting them in RAID-5 will cut back the performance considerably according to: PostgreSQL performance tuning. As the performance of databases greatly depends on the number of heads, the seemingly slow option B allows for 6x the amount of heads at the same price (not to mention the amount of available storage). Option A is about 12% cheaper compared to option C but is also 10 to 20% slower in database environments. Option (or using even less disks) should perform best when using e.g. two RAID-1 sets (one for OS+transaction logging, one for data).
--Lambertus 16:05, 21 March 2007 (UTC)

£10 bank fees on 50€ ??? wtf ? User:sxpert

Yes, my bank obviously charge a fixed fee for certain things, not a percentage. Swings and roundabouts. The 90E payment didn't attract any fee. In this case, a forex dealing fee. If the exchange was made at the sending bank, no fee would have been applied by my bank, but the sending bank may have levied a larger fee.NickH
there would be no change charge if you people would accept switching to the euro :D - User:Sxpert 21:54, 9 March 2007
That is waaaay off topic. -- User:NickH 23:23, 9 March 2007

Location-based partitioning

Is it possible to split the database server into machines serving different areas? The API could look-up latitude and longitude quickly and then send the data to the appropriate machine. Multiple machines tend to cheaper than one big machine. Is this more complicated than it sounds? -- User:Asc99c 20:47, 29 July 2007

Server Naming Theme Discussion and Vote

Voting has now closed

Results:

Cartographers : 12
Explorers : 11
Geographers : 9 
Topologists : 8
Famous Fictional Cats : 10
Fictional dragons ("here be dragons") : 18
Popular "amenity" tag values : 6 
GPS / GIS formats : 2
National Mapping Agencies : 2
Word "map" in different languages : 9
Capital cities : 10 


Our servers need new names. Its your turn to come up with an idea for a theme. We need to be able to pick nice simple easy to remember names from a common theme.

splitted up DBs for webservices?

Hi there, by developing an Geo app I noticed that the most apps would need the following informations:

  • area in boundingbox or polygone
  • where am I in the boundary hierachy?
  • roads
  • osmAPI/osmXAPI mirrors
  • diff tilemaps and their mirrors

By knowing this it would be possible to offer specialised webservices for example

  • areas.osm.org (name->boundingbox | polygon)
  • roads.osm.org (area->waynets)
  • isinm.osm.org (pos->boundary hierarchy)
  • packages.osm.org(name->.osm |.binoosm)

with the result that the main DB would be used only for real edits and the surrounding service DBs, which could be optimised for their tasks. What do you think about this suggestion? --!i! 10:38, 1 June 2009 (UTC)

wikimedia donated servers

Hi, I just wanted to ask about the recent announce that wikimedia foundation donated some servers to OSM. How many were donated, what will they do, where will they be hosted? Here's the link to the post about the donation: http://techblog.wikimedia.org/2010/02/wikimedia-donates-servers-to-deserving-non-profits/ --Jekader 23:17, 23 February 2010 (UTC)

hosted by OSM USA. Can ask if you have more questions -- Firefishy 23:38, 23 February 2010 (UTC)

Forum hosting, servers of local chapters

Hi there, would it be ok if we add the server of the forum right here? Oh and how can we link to the servers that had been donated to the local chapters (e.g. Germany: FOSSGIS/Server)? --!i! This user is member of the wiki team of OSM 18:22, 26 December 2010 (UTC)

Location + connection

Where exactly are the servers located and how fast connection do they have?--Kozuch 11:13, 23 September 2011 (BST)

Some are located at Imperial, some at UCL. These are two big universities in London, UK, with super-fast university bandwidth. If you're wondering about slow performance on any particular service (Platform Status) It's generally not a connection bandwidth issue -- Harry Wood 16:09, 23 September 2011 (BST)
"super-fast" is an interesting speed figure :-) --Kozuch 21:38, 23 September 2011 (BST)
We have multiple 100Mb/s links at UCL and multiple 1Gb/s links at IC. -- Firefishy 11:15, 27 September 2011 (BST)

Server Down

Suggest create the page ServerDown, to include info about:

Application error

The OpenStreetMap server encountered an unexpected condition that prevented it from fulfilling the request (HTTP 500)

Feel free to contact the OpenStreetMap community if your problem persists. Make a note of the exact URL / post data of your request.

This may be a problem in our Ruby On Rails code. 500 occurs with exceptions thrown outside of an action (like in Dispatcher setups or broken Ruby code)

--OpenBoy 09:01, 15 November 2011 (UTC)

There was a major rails port upgrade at around 9am (GMT/UTC) this morning. There may be a few moments of disruption this morning while some late bugs are resolved. -- Firefishy 09:58, 15 November 2011 (UTC)

Tile servers

Hi, I see that now we have some more tile servers. Can anybody give me a hint how they deal with the requests? Is there some kind of location based Load Balancing, or are they just another single Tile servers, that might get used if you use their dedicated tile URLs? --!i! This user is member of the wiki team of OSM 12:19, 2 November 2012 (UTC)

OSM uses a system called geodns - the request is directed via DNS to one of the Servers. See [2]. So no configuration on the client side is necessary. A russian user will automatically get the result from the cache/proxy in Moscow.--Zuse 20:56, 2 November 2012 (UTC)

TLS setup improvements?

Hi there, I did an simple SSL test and well, the results aren't that satisfying. Can you try to add *.osm.org to the certificate please? --KVLA-HRO-Mei (talk) 08:58, 5 February 2015 (UTC)

In-progress. https://github.com/openstreetmap/operations/issues/2 - Firefishy (talk) 09:13, 5 February 2015 (UTC)

tile server link broken?

https://dns.openstreetmap.org/tile.openstreetmap.org.html gives a grey background, but nothing else

  • That's for the old CDN and is now redundant. TomH (talk) 16:09, 17 July 2021 (UTC)