EmergencyEMERGENCY? Get 24/7 Help Now!

TokuDB Hot Backup Now a MySQL Plugin

 | February 5, 2015 |  Posted In: Tokutek, TokuView


In the recently released TokuDB 7.5.5 the implementation of TokuDB hot-backup moved from a patch to the MySQL Server, to MySQL Plugin.  Why did we make this change?

TokuDB hot backup makes a transactionally consistent copy of the TokuDB files while applications continue to read and write these files.  Christian Rober wrote a nice series of blogs about how hot backup works.  See TokuDB hot backup 1 and TokuDB hot backup 2 for details.  In summary, the TokuDB hot backup library intercepts system calls that write files and duplicates the writes on backup files. It does this while copying files to the backup directory.

There are two changes made to MySQL to get TokuDB hot backup working.

First, the hot backup library must be loaded into the mysqld server so that it can intercept system calls. One could use LD_PRELOAD to solve the system call intercept problem.  In contrast, Tokutek builds of MySQL link the hot backup library into mysqld so that is it automatically loaded when mysqld is started.

Second, there must be a user interface that can be used to start a backup, track its progress, and determine whether or not the backup succeeded.  We have patched the MySQL server to include a new SQL command (backup to ‘/my/backup/2015/2/1’) to initiate a backup.  We also defined some session variables that are used to monitor and control the backup.  The patch changed the SQL parser and added code to handle the new command.

We want to use TokuDB hot backup with enterprise AND community builds of MySQL, MariaDB, and Percona Server.  As a result, we replaced the patch with a plugin.  The patch would have been nearly impossible to get integrated into the various MySQL forks.

The TokuDB backup plugin kicks off a backup as a side effect of setting a backup session variable to the name of the destination directory.  In addition, we use session variables to monitor and control the backup process.  Here is how it works.  Lets suppose that we want to backup the MySQL data files into a backup directory called ‘/my/backup/2015/2/1’.

To limit the copy rate to 10 MB per second,  run the  ‘set tokudb_backup_throttle=10000000’ command.

To start the backup, run the ‘set tokudb_backup_dir=’/my/backup/2015/2/1’ command.  This command blocks until the backup is complete.  The entire backup is done by the check function attached to the ‘tokudb_backup_dir’ session variable.  It is really cool that we can execute code when a session variable is updated.

The backup progress can be monitored with ‘show processlist’ from another MySQL client.  The TokuDB hot backup will update the state of the backup periodically.

When the backup completes, the ‘set tokudb_backup_dir’ command returns.

Please refer to the TokuDB backup plugin github repository for further details.





  • Does this use cgroups like http://databaseblog.myname.nl/2014/11/throttling-mysql-enterprise-backup-with.html ?

    Why a variable and not and UDF?
    Then you could do something like:
    DO tokudb_backup_to(‘/my/backup/dir’);

    Also the COPYING file says GPLv2, but the plugin description says proprietary..

    • Tokudb hot backup does not use cgroups. Perhaps this is an interesting area to investigate.

      Why a variable and not a UDF? There are always many ways to implement a feature. This choice was arbitrary since I knew how to write the code for session variables. One can create another plugin that implements the tokudb backup API using a different choice.

      I will fixup the plugin description in the next release.

    • Tokutek hot backup consists of two components: the user interface and the hot backup library. The user interface was converted from a patch to MySQL to a MySQL plugin. The Tokutek hot backup library is unchanged and is only available in the Enterprise release.

  • Is relocation of the data files (datadir) when recovering via the hot backup allowed or are the tokudb files still fixed to the old datadir?

  • This still makes it tricky to get point in time backups from a slave. Short of completely stopping replication while the backup runs and more manually backing up myisam/innodb tables (which you need if nothing else for the mysql database).

Leave a Reply