Tiles@home/Server

Planned, current, and completed modifications to the Tiles@home website and its server

Planned

• Update client failure feedback system: NoData, BadUTF8, BadSVG, MissingFile, BlackTile
• When allowed, upgrade python-imaging on server to remove the gridpattern on lowzoom tiles
• Make lowzoom stitching faster by comparing if the four tiles is the same

In progress

Sebastian Spaeth is away until week 35

Historical

2008-07-15

Switched server to Tahngo implementation

2007-09-25

• As the API is no bottleneck anymore, disable the rate limiting. Much faster now. Implemented priority=3 for low priority bulk requests. Please make use of it.
• Low priority requests will get a queue waiting bonus of 1 per 48 hours now, so a prio=3 request will be rendered before a prio=2 (or 1) request if it was in the queue for a long time.

2007-09-11

• The server now only allows 10 high priority render requests per IP address. Subsequent requests will automatically be low-priority requests. Hopefully this let's us get "important" manual rerender requests go through before bulk requests take over everything.
• Allow only 50 active render requests, so they don't clogg up the API like they have been doing.

2007-09-01

• Changed the current client number to version 16, naming it Kingston. This contains the new Osmarender5 code and lowzoom capabilities. Osma5 is being used, while lowzoom is still off by default.

2007-08-08

• Modify request Queue handling. Changes:
  • A new request will only be taken if the same request is not in the PENDING, NEW, or ACTIVE state. This will hopefully prevent unneccessary double rendering of tiles
  • The same request can exist multiple times in the same state: This allows us to have the same (X,y) multiple times in the DONE state, if it actually has been rerendered within 48h. Thus our stats bookkeeping should be better now and correctly count them multiple times.
  • Added a colum "ip" to the request table, we will be able to capture the IP that started the request here and do rate-limiting per IP in the future. At the same time, shrank some other column fields, so total table size should be smaller.

2007-08-08

• Implement simple rate limiting. We only hand out 500 active requests at any one time. 2.5k clients were taking the API down.
• Only insert blank tile information when it actually provides new information. Delete all redundant blank tile entries, that will reduce the database from 135 mil entries by around 90%. This is a 4-day task or so.
• Reduced data base size by modifying the columns to be smaller.
• Have added some stats over the last weeks that document the health of the t@h server system.

2007-07-25

- change tiles_meta to InnoDB

• • Query seems to have stopped running, but the database is still MyISAM?!?
  • Someone's restarted the processing queue, 12pm 2007-07-26

2007-07-25

• Remove _details and _exists APIs

16:52, 5 July 2007 (BST)

• Auto re-request disabled

09:58, 30 June 2007 (BST)

• Block access to non-Jevier renderers, so that renderers compete equally for upload time

28 June 2007

• Queued uploads

11:53, 2 June 2007 (BST)

• Remove access to tiles on the old disk again, so that deleted tiles show up as deleted.

1 June 2007

New meta-info page /Tiles/info.php

29 May 2007

Reenabled access to tiles on old disk, so that map would look more complete for LinuxTag

15:01, 5 May 2007 (BST)

Dev server up, now with new disk space (not being used yet, it's on a separate partition)

Lots of version names are now blocked from uploading, due to Tiles@home/Dev/Coastline

18:55, 4 May 2007 (BST)

Dev server down

14:00, 4 May 2007

Requests queue turned off

10:00, 4 May 2007

Searching for all-sea tiles so that their image files can be deleted to save disk space. All 571-byte tiles have been identified, and a script is running to check which of those are sea.

Options for testing if a file is sea: (selected: MD5)

• Binary comparison (e.g. 'cmp' program)
• MD5_file from within PHP and compare to known hash
• Open file in imlib and test a pixel colour
• Run imagemagick compare with some options and parse the output
• Perl::GD compare function as used in lowzoom (problem: lack of Perl::GD library on server)

23:00, 3 May 2007

Maplint images at zoom 17 deleted to save disk space.

Disk usage drops to about 83%, enough for reiserfs to operate temporarily