Announcing TokuMXse v1.0.0-rc.0

UPDATE: Since the publication of this blog post, Tokutek subsequently published preliminary performance benchmark results for TokuMXse. In addition, we have recently released the fourth release candidate of TokuMXse for testing.  Also, since this post was written, MongoDB has decided to call the new release of MongoDB v3.0 (formerly known as v2.8).

Learn more about the performance benchmark; learn how to download RC 4 here.

The major work in MongoDB 2.8 is the creation of a “storage engine” API within the server.  This means that in the future, it will be much easier to create a product like TokuMX which uses a different storage implementation but keeps the same networking, clustering, and data modeling stack as MongoDB.  TokuMXse is the name of a new product we’ll add to the line, existing alongside TokuMX, which is an implementation of that storage engine interface using the same core TokuFT storage library as TokuDB and TokuMX.  We’ve written about some preliminary performance results comparing TokuMXse to TokuMX.

First things first: you can learn how to download TokuMXse v1.0.0-rc.4 here.

The pros and cons of a storage engine interface are complicated, but it comes down to compatibility versus innovation.  TokuMXse, since its code is sequestered behind an API which is intended to be static, should be able to follow future upstream development with relative ease; in comparison, since TokuMX is a fork in which we’ve made changes to many parts of the codebase, it takes much more work to incorporate upstream development, and so TokuMX lags behind MongoDB development.  However, behind this small API, there is a limited set of improvements we can make.  TokuMX features like clustering indexes, primary keys, read-free replication and sharding migrations, fast updates, and multi-document ACID semantics require changes in other parts of the codebase, and thus can’t be implemented from within the storage engine API as it is today.  This is likely to remain the case for most such features.

So, long story short, TokuMXse is MongoDB 2.8-compatible, and has the enhanced speed, concurrency, and compression of Fractal Tree storage, but won’t have all of the advanced features of TokuMX.  It’s a good choice for applications which require the latest MongoDB features and just want reliably high performance and compression.

When running the server, make sure to start mongod with --storageEngine=tokuft, and check the additional options starting with --tokuft in the --help text. There is also a tokuft section of the db.serverStatus() output and in db.collection.stats(). Let us know what you think on the tokumx-user mailing list, or on JIRA.

Share this post

Comments (5)

  • Vlad

    Thanks for creating this! The only reason I have a preference for TokuMXse over TokuMX is to provide compatibility with the Meteor framework, which depends on the MongoDB Oplog for triggering reactive events.

    TokuMX implements a different format than MongoDB does for its Oplog, so Meteor cannot parse it.

    Tokutek needs to either implement a configuration option in TokuMX to generate the original MongoDB-format Oplog, or to add support to Meteor for parsing the TokuMX-format Oplog.

    I think it’s worthwhile for you to pay attention to Meteor, because its been rapidly growing in popularity and I expect it to be significant driving force in propelling demand for MongoDB – the database that Meteor is currently bundled with and natively supports.

    I plan to test TokuMXse with Meteor over the next month to see how well they interoperate.

    April 29, 2015 at 10:43 am
    • Jon Tobin

      Thanks for input! We’ll definitely keep an eye on Meteor and would love to hear more from the Meteor community on interest in compatibility with TokuMX. Ultimately, the MongoDB community as a whole drives the roadmap for TokuMX.

      April 29, 2015 at 11:03 am

Comments are closed.