October 1, 2014

Percona Server with TokuDB: Packing 15TB into local SSDs

Two weeks ago we released an Alpha release of Percona Server with TokuDB. Right now I am on a final stage of evaluation of TokuDB for using in our project Percona Cloud Tools and it looks promising.

What is the most attractive in TokuDB? For me it is compression, but not just compression: TokuDB provides great performance over compressed data.

In my synthetic tests I saw a compression ratio of 10:1 (TokuDB LZMA to InnoDB uncompressed), in the real production data it is less, 6:1, but still impressive.

In our servers we have 4 x SSD Crucial M500 960GB combined in RAID5, which give 2877.0 GB of usable space. With TokuDB we should be able to pack around 15TB of raw data. Of course we can try InnoDB compression, but the best we can get is 2x compression without sacrificing performance.

And of course TokuDB is transaction, fully ACID-compliant with automatic crash-recovery storage engine.

This all makes TokuDB a very attractive choice for handling terabytes of data (or as it popular to say nowadays “BigData”).

One of first operational questions we have is how to handle backups.
For backups we use LVM partitions and the mylvmbackup tool. Unfortunately Percona XtraBackup is not able to handle TokuDB tables (and probably won’t be able anytime soon). The other choice is to use TokuDB Hot back-up, available by Tokutek Enterprise Subscription. I did not test it myself, so I can’t provide any feedback.

And of course there are things which I do not fully like in TokuDB:

  • No Foreign Keys support. It is not a big issue for us, but I know for some users this is a showstopper.
  • Time-based checkpoints. You may not notice a direct effect from this, but we clearly see it in our benchmarks. Every 60 sec (default timeperiod between checkpoints) we see a drop in throughput during write-intensive benchmarks. It is very similar to drops in InnoDB we tried to solve (and are still trying), for example see Adaptive flushing in MySQL 5.6. My advice to the Tokutek team would be to also look into a fuzzy check-pointing, instead of time-based.
  • All TokuDB files are stored in a single directory, sometime with mangled filenames. This especially becomes bad in sharding or multi-tenant environments when tens of thousands of files are in the same directory

Well, I guess for now, we will take these limitations as TokuDB specific and will deal with them.

Next week we plan on a Beta release of Percona Server with TokuDB, so stay tuned!

About Vadim Tkachenko

Vadim leads Percona's development group, which produces Percona Clould Tools, the Percona Server, Percona XraDB Cluster and Percona XtraBackup. He is an expert in solid-state storage, and has helped many hardware and software providers succeed in the MySQL market.

Comments

  1. For deployments using flash the bytes-written rate can be very important and TokuDB should be writing less to storage than InnoDB. I hope you join me in reporting write-amplification for benchmarks — http://smalldatum.blogspot.com/2014/03/insert-benchmark-for-innodb-mongodb-and.html

  2. Mark,

    That’s absolutely correct. I indeed should have mentioned that, but by some reason I have not.
    TokuDB writes much less data, that is a benefit for a health of SSD.

  3. Mark,

    Right, I think it would be very interesting to show this data from the real world examples. It is worth to note one can look at this both as amount of logical writes which TokuDB is doing as well as difference to write amplification on the hardware level. The larger blocks size TokuDB uses should put less pressure on garbage collection and reduce write amplification though which would be interesting to check out.

  4. I have been using iostat rates but for flash devices getting the bytes written reported by the device which includes GC overheads is even more interesting. This isn’t a widely appreciated topic and I think that the expertise of Tokutek & Percona can help people to understand it.

  5. One of my clients was internally evaluating Toku for a long time because of the possibility of easy compression. Unfortunately it had a bug, where after been put to use for a while would start dumping massive amounts of temp files into the mysql directory (or another directory with the appropriate settings). Toku is also kind of advertised a a zero conf engine, so they don’t encourage tinkering with them, so it would be really great to see if you could get some data on that. That and high compression mode was way to slow to even bother with while the default compression mode was actually nice.

    I might look at this engine again Percona gives it s gold star approval and some guidelines based on actual use. Its clearly s long term winner if they can get it where it needs to be,

Speak Your Mind

*