Embracing MongoRocks

MongoRocksThis blog post discusses Percona’s future plans for TokuMX, PerconaFT, and MongoRocks for MongoDB 3.2 and later releases.

At Percona, we focus on providing the best solution for our customers, rather than falling into “big ego,” “not invented here” or “sunk cost fallacy” messages. We have limited engineering resources and want to concentrate on the things that matter the most for our community and our customers – the things that make a positive difference.

When we acquired Tokutek, TokuMX was a product that truly made a difference for a lot of MongoDB users. It allowed them to do what they could not do with MongoDB. However, things in the technology world change rapidly, as they have in the year since the acquisition.   

During the last year, the WiredTiger storage engine became generally available and demonstrated great performance for many read-focused and in-memory workloads. In addition, the RocksDB-based MongoRocks storage engine is well-optimized for handling large data write-intensive workloads. 

(You can see some of the benchmarking we’ve done to verify the performance of these engines here.)

With these developments, Tokutek’s Fractal Tree storage engine (now named PerconaFT) lost major performance advantages for the most common workloads. Additionally, the fact that MongoDB internally uses an optimistic locking pattern, while PerconaFT was designed with a pessimistic locking pattern, means that it would take a lot of work to make PerconaFT the best solution in a MongoDB environment.

As result, we have decided to deprecate PerconaFT in Percona Server for MongoDB 3.2, with plans of removing it in Percona Server for MongoDB 3.4.  At the same time, we will move our focus towards working together with Facebook on making MongoRocks the leading write-optimized storage engine for the MongoDB ecosystem.    

We will focus on fixing any remaining feature and performance gaps to ensure moving from PerconaFT to MongoRocks with Percona Server for MongoDB is as simple as possible – and results in even better performance.

With the release of Percona Server for MongoDB 3.2, we’re fully embracing MongoRocks and lifting its experimental status to be fully supported for all production environments.

You may ask, what does this mean for TokuDB? This means our Fractal Tree Indexing engineering team can focus on MySQL workloads rather than supporting two different database servers that have very different expectations from the underlying storage engine, and will allow us to improve TokuDB faster than before.

Share this post

Comments (5)

  • Andy

    If RocksDB has a performance advantage over PerconaFT, does that mean MyRocks also has a performance advantage over TokuDB? Between MyRocks & TokuDB which one would you recommend?

    June 2, 2016 at 5:49 am
  • Ernie S.

    @Andy: MyRocks isn’t production ready. TokuDB has been run in production for years by various folks. So, at least right now, I think any performance differences that may or may not exist are irrelevant.

    June 2, 2016 at 12:47 pm
  • Peter Zaitsev

    Ernie, Thank you for your comment here.

    Indeed it is not reasonable to make comparison between MyRocks and TokuDB until MyRocks has been in production for some time.

    It is also worth to note what as much as it might look from the surface what below any database engine there is pretty simple data store, in fact they get to be integrated pretty tightly.

    TokuDB was designed for MySQL Storage Engine API from the very beginning. Some properties of this API such as MVCC and Pessimistic lock handling was done in TokuMX, this is however different from the way MongoDB is using storage engine. As result these and other issues MongoDB is not driving PerconaFT in a way to realize its full potential.

    June 2, 2016 at 2:34 pm
  • Vlad

    I welcome Percona’s renewed focus on continued development of the TokuDB storage engine for MySQL.

    Since Percona’s acquisition of Tokutek, it felt like TokuDB development ground to a halt and perhaps the work on PerconaFT for MongoDB was the distraction.

    Since MySQL 5.7 was released, InnoDB was enhanced to support very useful features including like Indexable Virtual Columns (also called Generated Columns) and Spatial Indexing. InnoDB has also added support for page compression.

    To me, these improvements to InnoDB have resulted in TokuDB losing some of its edge. However, TokuDB still has the advantage of online addition/removal of columns and the benefits of the TokuDB’s fractal tree indexes.

    Implementing TokuDB support for the missing features Indexable Virtual Columns (Generated Columns), Full-Text indexing and Spatial indexing (in order of priority) would make TokuDB a top choice again.

    July 11, 2016 at 9:42 am
  • Alex

    I have used TokuMX as my main storage engine years ago, and was profit from both multi clustering index and excellent compress ratio. That helps us can use a little storage to serve a lot of different range queries – by using multi clustering index with gzip compressor.
    Does new release of Percona Server for MongoDB can still support this scenario?

    November 28, 2016 at 9:55 pm

Comments are closed.

Use Percona's Technical Forum to ask any follow-up questions on this blog topic.