Migrate from TokuMX to Percona Server for MongoDB

Percona no longer develops or supports TokuMX software. The product has been EoL’d. We recommend TokuMX users migrate to Percona Server for MongoDB.

This blog post details how to migrate from TokuMX to Percona Server for MongoDB.

As part of our ongoing plans to embrace the MongoDB community, we have increased support and software around MongoDB. Percona’s last release of the TokuMX software platform was on September 15th, 2015.  Percona is announcing a General EOL of TokuMX Support as of May 1st, 2017.

If you are currently a MongoDB Support customer running TokuMX, we highly recommend migrating to Percona Server for MongoDB 3.4.x (and the WiredTiger or MongoRocks storage engine). We would be happy to assist in this migration through our regular Support and Consulting communications channels.

A Brief History of TokuMX

A group of engineers at TokuTek started the TokuMX project. The TokuFT fractal tree engine, now called PerconaFT, was already implemented as the MySQL storage engine plug-in known as TokuDB. The group needed new database technologies where a fractal tree might be of use. MongoDB 2.2 had serious performance limitations and inadequate concurrency and durability, making it a good candidate. The team committed the first work on TokuMX to the source tree in September of 2012. TokuMX v1.0 was released by June 2013.

When TokuMX was first implemented, there was no storage engine interface in the MongoDB code. This meant that the integration of the fractal tree required considerable changes to the base code of MongoDB 2.x. They made very impressive performance gains and implemented a full SQL-style stateful transaction interface. This allowed multi-operation, multi-document, and multi-collection concurrent transactions to remain consistent.

In mid-2014, MongoDB, Inc. began work on a storage engine API in the 2.7 development code base. A few months later, the MongoDB storage engine implementation of TokuFT began as a project known as TokuMXse. This became the basis for the PerconaFT storage engine in Percona Server for MongoDB 3.0. During this time, MongoDB, Inc. also worked closely with WiredTiger to implement their high-speed storage engine which also provided better concurrency and durability. In December of 2014, MongoDB, Inc. acquired WiredTiger. The WiredTiger storage engine was released with MongoDB 3.0.0 a few months later in March 2015.

The reason Percona Server for MongoDB does not implement stateful transactions:

Unfortunately, some core design decisions in MongoDB 3.x made it impractical to implement the stateful transactions that SQL databases and TokuMX have. The MongoDB 3.x storage engine API is very closely tied to a concurrency model known as optimistic concurrency. This is implemented using C++ exceptions which throw and retry a single operation until it doesn’t conflict with another. This strategy has advantages and disadvantages. Developers don’t need to worry about implementing advanced transaction handlers for deadlocks and rollbacks. However, MongoDB doesn’t support transactions that involve multiple operations, documents or collections defined and controlled by the user application. MongoDB can’t implement this without far-reaching modifications to the MongoDB code base.

Without a major redesign of the MongoDB 3.x storage engine layer and the higher level database code, SQL style stateful transactions will not be possible. The MongoDB 3.x design also dramatically reduced the performance of the fractal tree engine which is designed around the traditional ACID model and does not support optimistic concurrency. As a result, we deprecated the fractal tree engine (PerconaFT) and removed it from Percona Server for MongoDB 3.4.

Performing a Migration

There are a few things to consider before beginning migration. Percona Server for MongoDB doesn’t implement a few TokuMX features.

Some TokuMX commands that are not available in PSMDB

  •  Stateful transaction commands
    • beginTransaction
    • rollbackTransaction
    • commitTransaction
  • Bulk loader commands
    • beginLoad
    • commitLoad
    • abortLoad
  • Partition commands
    • addPartition
    • dropPartition
    • getPartitionInfo
    • replAddPartition
    • clonePartitionInfo

If your application depends on these features, migration may require some changes to function correctly.  You will also want to review the upstream MongoDB 2.x to 3.x documentation as well as the custom commands section of the TokuMX documentation to identify differences between the versions.

There are two methods to migrate your data from TokuMX to Percona Server for MongoDB. We’ve documented the methods in a Percona Lab GitHub repository.

Migrate from TokuMX to Percona Server for MongoDB

If you have any questions or concerns about your TokuMX to Percona Server for MongoDB migration, please contact Percona Support.  We are here to help 24x7x365 online or by phone.

Share this post