EmergencyEMERGENCY? Get 24/7 Help Now!

TokuMX 2.0 preview and bug hunt

 | September 22, 2014 |  Posted In: Tokutek, TokuView

PREVIOUS POST
NEXT POST

TokuMX 2.0 is coming at the end of this month, and it’s going to be the most innovative release of TokuMX since our first release last June.  Version 2.0 represents the realization of many of our engineering aspirations from when we started the TokuMX project, and we’re all eager to get it in your hands.

Some of the new features we’re particularly excited about include:

  • Fast updates that can boost the speed of updates to unindexed fields by more than 10x.
  • Ark elections making replication trustworthy and more ops-friendly.
  • Smaller oplog entries made possible with a brand-new rollback algorithm.
  • Geospatial compatibility with haystack, 2d, and 2dsphere indexes and queries in MongoDB.

We’ll be writing more about these and other changes in the coming weeks, so stay tuned.

We also want to make sure 2.0 is the highest quality it can be, so we’re announcing a bug hunt for TokuMX 2.0.  It’s pretty simple: download and run our preview release of TokuMX 2.0, find a bug in it, report it to our JIRA, and we’ll send a t-shirt to the address of your choice.

To facilitate this, we’re also releasing documentation of these new features and behaviors, including a limited form of partitioned collections that may also be sharded, in our TokuMX 2.0 preview documentation.  Make sure you check out the new sections on fast updates, replication rollback, and sharded and partitioned collections.

Linux binaries of the preview release are available here, and they’re also available in our testing deb repositories for Debian and Ubuntu.

The bug hunt ends Friday, September 26th at 11:59PM EDT.  Good luck and have fun!

PREVIOUS POST
NEXT POST

6 Comments

  • Thanks for the update. We are looking forward to a lot of these features, especially the boost in the update speeds.

    I see that you’re altering the oplog strategy, my question is whether it would be safe for us to deploy the new binary on a secondary replica set member to test while leaving the current binary on the rest of the replica set members? Would the changes in how the oplog is handled effect replication in this case in any way?

    Also, I’m wondering if changes to the oplog (or any other portion of this release) would effect our ability to rollback out of this binary and back to current, production ready binary if we found a bug that made us have to rollback?

    If the new 2.0 binary is perfectly compatible running alongside or interchanging it with the current binary, I’ll be very happy to test it out on our servers, starting as a secondary and rolling up to primaries if we experience no production threatening bugs, but if there are any potential conflicts I would appreciate a mention of what to watch for and a recommended strategy for rollback.

    • It should be safe to deploy as a test secondary, but keep in mind that the binary is not ready for production. Just make sure that it cannot become a primary, by setting its priority to 0. The oplog version, which can be seen in rs.status(), is higher than in previous versions. As a result, it will sync off other 1.5 members, but no 1.5 members will sync off of it. That is why you do not want it to become primary.

      As for rollback, unfortunately, we don’t support rollback from 2.0 to 1.5. The best way to rollback is to have a 1.5 backup of a member, and a 1.5 member of the replica set. Should something go wrong, you can restore everything to 1.5.It should be safe to deploy as a test secondary, but keep in mind that the binary is not ready for production. Just make sure that it cannot become a primary, by setting its priority to 0. The oplog version, which can be seen in rs.status(), is higher than in previous versions. As a result, it will sync off other 1.5 members, but no 1.5 members will sync off of it. That is why you do not want it to become primary.

      As for rollback, unfortunately, we don’t support rollback from 2.0 to 1.5. The best way to rollback is to have a 1.5 backup of a member, and a 1.5 member of the replica set. Should something go wrong, you can restore everything to 1.5.

  • “Updates that specify {upsert: true}. Such an update needs to modify secondary indexes if the document doesn’t yet exist.”
    … what about upserts without secondary-indexes ?

    “In general, only updates that can modify the primaryKey without modifying indexes can be fast, because modifying indexes requires reading the full document to identify what the appropriate index entries are.”

    Shouldn’t it be “read” the primaryKey ?

    • Making upserts without secondary indexes fast are not supported yet. It is something we can do in the future without much difficulty. We don’t currently have a good handle on the demand for fast upserts without secondary indexes. Do you have an application that can benefit from such a feature?

      And good catch on the documentation, it should be “only updates that can modify the document without modifying indexes”.

      • Fast/non-reading updates are the same idea/feature-request that I thought for hypertable (they have non-reading counters):
        https://groups.google.com/forum/#!searchin/hypertable-user/document$20cells/hypertable-user/l9nWyg-_i40/4SEdNsONhxEJ

        I’m building an analytics app where, > 99% of my queries are upserts.

        With this I can move some of my queries to normal updates, and keep ~33% to upserts. But I’m not experiencing high traffic yet.

        • I’d be interested in hearing more about this application. Would you care to share details on the tokumx-user google groups?

Leave a Reply