First thoughts regarding the MySQL Falcon storage engine
One of my DBA colleagues mentioned that MySQL has released the first alpha version of the Falcon storage engine, which is advertised to most efficiently utilise modern hardware to provide a high-performance scalable replacement for InnoDB, which MySQL naturally tries to reduce dependency of.
Unfortunately, just based on reading the Falcon documentation, I must draw the conclusion that without extensive further development, it won't be usable for very large installations such as the ones we run for Habbo for a number of reasons. I'm usually much more positive about MySQL, it after all being technology that has enabled Habbo to grow more than 100% every year I've been working on it, but this is a disappointment.
It supports just one tablespace per database, and each tablespace stores all data in a single file. While the concurrency problems of single-file access can be eliminated with careful application of modern kernel, filesystem and disk subsystem technology, single file databases still suffer from major administration issues.
Since a database can't be extended by additional tablespaces and data migrated by the storage engine, you'd better trust your capacity to indefinitely increase available storage space under one filesystem or downtime can't ever become a problem for you. Don't even think about deploying Falcon without a high-end NAS device that supports many times your current storage requirements, reliable logical volume management and an extendable file system. A database is also limited by the filesystem's maximum file size, so make sure that won't be a problem either. I wouldn't recommend ext3 for Falcon.
You'll also need to make backups either via SQL dumping the entire database (not really feasible for daily routine) or by backing up a single file, so either your filesystem, LVM system or storage device must support snapshot backups. Scalability may still become an issue, so be sure that the approach you choose doesn't degrade performance as file size grows.
Just one thread writing to disk may at first blush sound like excellent
performance maximisation technique, but it forces you to make a choice between
reliability (since it applied to log writes too, transactions are committed to
disk in a serialized fashion - no concurrency) and scalability ("commits" to
ram cache and background disk flushes certainly will perform well and scale
nicely, but what if there's a power failure?). And this is not even the road to
highest possible performance - the highest-end disk subsystems will become CPU
limited if only one thread will be able to send I/O requests.
With one table space comes one cache/buffer pair, so developers are either forced to split their data model to multiple logical databases or suffer under one unpartitionable system where one bad table scan by one part of the application wipes the buffers from underneath the entire application. A truly modern storage system permits the DBA to assign certain tables or indices to their own caches and buffer spaces and retain a single logical model for software developers. MySQL has never had this ability, and apparently Falcon won't bring it, either.
A more traditional DBA might also cringe at the statement "it is impossible to predict or calculate the disk storage space required for a specific dataset." Many, many complaints could be made about the alpha-release's other restrictions, but I'll give MySQL the chance to keep their promise to address them in forthcoming versions.
I don't really understand which of its features qualify it as technology that utilises modern computers to the best possible effect. Perhaps they're referring to it automatically compressing data on disk? Sure, that may be useful, but it may just as well become a bottleneck when single-row updates require entire pages to be recompressed. Just that feature alone doesn't impress me. It's not more easily administrated, nor does it (on paper at least) address this kind of performance issues. At best, it's an upgrade to MyISAM, but shouldn't be mistaken for a solution to high-performance transactional database requirements.
More on it once I've had a chance to do some practical experimentation (might be a while).
Update: It seems Peter Zaitzev has benchmarked Falcon against MySQL's other storage engines, verifying my suspicion that it doesn't scale properly. Do note that neither MyISAM nor InnoDB show ideal scaling performance either.