Talk:SQLite

failing on large datasets?

Well, this needs work, especially the unsupported assertion about failing on large datasets. Works fine for me on multi-gigabyte datasets. --Scruss (talk) 13:44, 26 August 2013 (UTC)

I completely agree. The message on the wiki page is directly wrong. Many XML parsers fail with memory errors before SQLite does. I have a 1.345 Tb SQLite database on a USB 3.0 disk and there are no problems here.

To quote the SQLite documentation:

13. Maximum Number Of Rows In A Table

The theoretical maximum number of rows in a table is 264 (18446744073709551616 or about 1.8e+19). This limit is unreachable since the maximum database size of 140 terabytes will be reached first. A 140 terabytes database can hold no more than approximately 1e+13 rows, and then only if there are no indices and if each row contains very little data.

14. Maximum Database Size

Every database consists of one or more "pages". Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).

This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

I would argue that a poor configuration or schema implementation is more likely to cause problems.

Here are some hints for speeding up the usage of SQLite with Python: http://codereview.stackexchange.com/questions/26822/myth-busting-sqlite3-performance-w-pysqlite

-- 09:10, 6 April 2015‎ User:Root-11

Well given the bad english, the sentence seemed highly dubious to me before I even read this. Deleting it now

You're obviously very knowledgeable about this stuff. Please feel free to be bold in fixing stuff like this! (Wiki Help#Getting started with wiki editing)

-- Harry Wood (talk) 10:24, 14 July 2015 (UTC)