The Future of TokuDB at PerconaPeter Zaitsev
As soon as we announced the fact that MyRocks was coming to Percona Server for MySQL at Percona Live Europe, rumors appeared that this means we’re going to phase out TokuDB at the same time.
I can understand why those rumors would start: just a few months ago we deprecated Fractal Trees technology (called PerconaFT) in favor of MongoRocks and RocksDB for Percona Server for MongoDB.
As much as this might look like the same situation as with PerconaFT, TokuDB is very different. PerconaFT was actually a new port of Fractal Trees technology for MongoDB. It used the MongoDB storage engine API and focused on replacing TokuMX (a full fork of an earlier version of MongoDB). PerconaFT was new, and MongoRocks was already well-established and battle-tested by Parse. The MongoDB internal design was also much more friendly towards the RocksDB storage engine than Fractal Trees technology (as David Murphy explains here).
Our research revealed that PerconaFT had very few current users, and MongoRocks was a superior choice for new users. This is why we depreciated it.
For MySQL, TokuDB is a rather mature storage engine. It has been in production for more than five years and is trusted by many businesses and large enterprises (in industries ranging from banking and finance to gaming) for use with critical applications.
MyRocks however, with as much promise as it shows, has not been extensively battle-tested outside of Facebook. And as it is hard to beat “Facebook scale,” it hasn’t exposed its performance under a large variety of workloads. There are also some known limitations in MyRocks that need work.
Ultimately though, I believe MyRocks has a great future, and we at Percona are going to take part in making this future reality. But this does not mean we plan to immediately abandon TokuDB, or place it on “life support”.
You can see new development going on for TokuDB: we have dramatically improved block allocation performance and added a new, more convenient TokuDB file layout option. We are also working on adding support for new compression algorithms in TokuDB, support for Performance Schema and refactoring the checkpoint process to ensure uniform performance on all workloads. It is true TokuDB is not changing as rapidly as RocksDB, but that is to be expected – as the more mature storage engine, it should embrace stability first and foremost.
Our overall principle at Percona is to keep our ego in check. We want to provide the best open source ecosystem solutions available to our customers, whether they were developed in-house or not. This means that within 3-5 years MyRocks could possibly be superior to TokuDB in functionality and performance for all imaginable workloads. Or an even better open source storage engine technology might get developed and come to market. If or when this happens, we will consider depreciating TokuDB in favor of such technology – while providing the customers with enough notice, tools and support to ensure they can successfully migrate.
What this really means is that TokuDB is still being actively supported and developed for at least Percona Server 5.7 and Percona Server 8.0 (and longer). We will continue this as long as a reasonable use case remains for this technology.
Please feel free to contact us with any questions, or add them in the comments below.