TokuDB Hot Backup Now a MySQL Plugin

In the recently released TokuDB 7.5.5 the implementation of TokuDB hot-backup moved from a patch to the MySQL Server, to MySQL Plugin.  Why did we make this change?

TokuDB hot backup makes a transactionally consistent copy of the TokuDB files while applications continue to read and write these files.  Christian Rober wrote a nice series of blogs about how hot backup works.  See TokuDB hot backup 1 and TokuDB hot backup 2 for details.  In summary, the TokuDB hot backup library intercepts system calls that write files and duplicates the writes on backup files. It does this while copying files to the backup directory.

There are two changes made to MySQL to get TokuDB hot backup working.

First, the hot backup library must be loaded into the mysqld server so that it can intercept system calls. One could use LD_PRELOAD to solve the system call intercept problem.  In contrast, Tokutek builds of MySQL link the hot backup library into mysqld so that is it automatically loaded when mysqld is started.

Second, there must be a user interface that can be used to start a backup, track its progress, and determine whether or not the backup succeeded.  We have patched the MySQL server to include a new SQL command (backup to ‘/my/backup/2015/2/1’) to initiate a backup.  We also defined some session variables that are used to monitor and control the backup.  The patch changed the SQL parser and added code to handle the new command.

We want to use TokuDB hot backup with enterprise AND community builds of MySQL, MariaDB, and Percona Server.  As a result, we replaced the patch with a plugin.  The patch would have been nearly impossible to get integrated into the various MySQL forks.

The TokuDB backup plugin kicks off a backup as a side effect of setting a backup session variable to the name of the destination directory.  In addition, we use session variables to monitor and control the backup process.  Here is how it works.  Lets suppose that we want to backup the MySQL data files into a backup directory called ‘/my/backup/2015/2/1’.

To limit the copy rate to 10 MB per second,  run the  ‘set tokudb_backup_throttle=10000000’ command.

To start the backup, run the ‘set tokudb_backup_dir=’/my/backup/2015/2/1’ command.  This command blocks until the backup is complete.  The entire backup is done by the check function attached to the ‘tokudb_backup_dir’ session variable.  It is really cool that we can execute code when a session variable is updated.

The backup progress can be monitored with ‘show processlist’ from another MySQL client.  The TokuDB hot backup will update the state of the backup periodically.

When the backup completes, the ‘set tokudb_backup_dir’ command returns.

Please refer to the TokuDB backup plugin github repository for further details.



Share this post

Comments (8)

  • Daniël van Eeden Reply

    Does this use cgroups like ?

    Why a variable and not and UDF?
    Then you could do something like:
    DO tokudb_backup_to(‘/my/backup/dir’);

    Also the COPYING file says GPLv2, but the plugin description says proprietary..

    February 7, 2015 at 7:31 am
    • Rich Prohaska Reply

      Tokudb hot backup does not use cgroups. Perhaps this is an interesting area to investigate.

      Why a variable and not a UDF? There are always many ways to implement a feature. This choice was arbitrary since I knew how to write the code for session variables. One can create another plugin that implements the tokudb backup API using a different choice.

      I will fixup the plugin description in the next release.

      February 9, 2015 at 8:54 am
  • leafonsword Reply

    Since hot backup is available as open source, what’s the advantage of TokuDB Enterprise Edition?

    March 27, 2015 at 4:30 am
    • Rich Prohaska Reply

      Tokutek hot backup consists of two components: the user interface and the hot backup library. The user interface was converted from a patch to MySQL to a MySQL plugin. The Tokutek hot backup library is unchanged and is only available in the Enterprise release.

      March 27, 2015 at 9:48 am
      • Denis Reply

        So I can’t use free hot backup plugin until I purchase Enterprise Edition

        February 19, 2017 at 5:17 am
  • Alex Reply

    Is relocation of the data files (datadir) when recovering via the hot backup allowed or are the tokudb files still fixed to the old datadir?

    May 12, 2015 at 11:15 am
  • Denis Raguzov Reply

    So I can’t use Toku hot backup with free version ?

    December 9, 2015 at 12:28 pm
  • Mike Siekkinen Reply

    This still makes it tricky to get point in time backups from a slave. Short of completely stopping replication while the backup runs and more manually backing up myisam/innodb tables (which you need if nothing else for the mysql database).

    November 29, 2016 at 1:32 pm

Leave a Reply