TokuDB file management¶

Moving TokuDB data files to a location outside of the default MySQL datadir¶

TokuDB uses the location specified by the tokudb_data_dir variable for all of its data files. If the tokudb_data_dir variable is not explicitly set, TokuDB will use the location specified by the servers datadir for these files.

The TokuDB data files are protected from concurrent process access by the __tokudb_lock_dont_delete_me_data file that is located in the same directory as the TokuDB data files.

TokuDB data files may be moved to other locations with symlinks left behind in their place. If those symlinks refer to files on other physical data volumes, the tokudb_fs_reserve_percent monitor will not traverse the symlink and monitor the real location for adequate space in the file system.

To safely move your TokuDB data files:

1. Shut the server down cleanly.
2. Change the tokudb_data_dir in your my.cnf configuration file to the location where you wish to store your TokuDB data files.
3. Create your new target directory.
4. Move your *.tokudb files and your __tokudb_lock_dont_delete_me_data from the current location to the new location.
5. Restart your server.

Moving TokuDB temporary files to a location outside of the default MySQL datadir¶

TokuDB will use the location specified by the tokudb_tmp_dir variable for all of its temporary files. If tokudb_tmp_dir variable is not explicitly set, TokuDB will use the location specified by the tokudb_data_dir variable. If the tokudb_data_dir variable is also not explicitly set, TokuDB will use the location specified by the servers datadir for these files.

TokuDB temporary files are protected from concurrent process access by the __tokudb_lock_dont_delete_me_temp file that is located in the same directory as the TokuDB temporary files.

If you locate your TokuDB temporary files on a physical volume that is different from where your TokuDB data files or recovery log files are located, the tokudb_fs_reserve_percent monitor will not monitor their location for adequate space in the file system.

To safely move your TokuDB temporary files:

1. Shut the server down cleanly. A clean shutdown will ensure that there are no temporary files that need to be relocated.
2. Change the tokudb_tmp_dir variable in your my.cnf configuration file to the location where you wish to store your new TokuDB temporary files.
3. Create your new target directory.
4. Move your __tokudb_lock_dont_delete_me_temp file from the current location to the new location.
5. Restart your server.

Moving TokuDB recovery log files to a location outside of the default MySQL datadir¶

TokuDB will use the location specified by the tokudb_log_dir variable for all of its recovery log files. If the tokudb_log_dir variable is not explicitly set, TokuDB will use the location specified by the servers datadir for these files.

The TokuDB recovery log files are protected from concurrent process access by the __tokudb_lock_dont_delete_me_logs file that is located in the same directory as the TokuDB recovery log files.

TokuDB recovery log files may be moved to another location with symlinks left behind in place of the tokudb_log_dir. If that symlink refers to a directory on another physical data volume, the tokudb_fs_reserve_percent monitor will not traverse the symlink and monitor the real location for adequate space in the file system.

To safely move your TokuDB recovery log files:

1. Shut the server down cleanly.
2. Change the tokudb_log_dir in your my.cnf configuration file to the location where you wish to store your TokuDB recovery log files.
3. Create your new target directory.
4. Move your log*.tokulog* files and your __tokudb_lock_dont_delete_me_logs file from the current location to the new location.
5. Restart your server.

Improved table renaming functionality¶

When you rename a TokuDB table via SQL, the data files on disk keep their original names and only the mapping in the Percona FT directory file is changed to map the new dictionary name to the original internal file names. This makes it difficult to quickly match database/table/index names to their actual files on disk, requiring you to use the INFORMATION_SCHEMA.TOKUDB_FILE_MAP table to cross reference.

Beginning with Percona Server for MySQL 5.6.33-79.0 a new server option was introduced called tokudb_dir_per_db to address this issue.

When tokudb_dir_per_db is enabled (OFF by default), this is no longer the case. When you rename a table, the mapping in the Percona FT directory file will be updated and the files will be renamed on disk to reflect the new table name.

Improved directory layout functionality¶

Many users have had issues with managing the huge volume of individual files that TokuDB and Percona FT use.

Beginning with Percona Server for MySQL 5.6.33-79.0 a new server option was introduced called tokudb_dir_per_db to address this issue.

When tokudb_dir_per_db variable is enabled (OFF by default), all new tables and indices will be placed within their corresponding database directory within the tokudb_data_dir or server datadir.

If you have tokudb_data_dir variable set to something other than the server datadir, TokuDB will create a directory matching the name of the database, but upon dropping of the database, this directory will remain behind.

Existing table files will not be automatically relocated to their corresponding database directory.

You can easily move a tables data files into the new scheme and proper database directory with a few steps:

mysql> SET GLOBAL tokudb_dir_per_db=true;
mysql> RENAME TABLE <table> TO <tmp_table>;
mysql> RENAME TABLE <tmp_table> TO <table>;

Editing TokuDB files with tokudb_dir_cmd¶

In Percona Server for MySQL 5.6.36-82.0 new tokudb_dir_cmd variable has been implemented that can be used to edit TokuDB files. WARNING: Use this variable only if you know what you’re doing otherwise it WILL lead to data loss.

This method can be used if any kind of system issue causes the loss of specific .tokudb files for a given table, because the TokuDB tablespace file mapping will then contain invalid (nonexistent) entries, visible in INFORMATION_SCHEMA.TokuDB_file_map table.

This variable is used to send commands to edit directory file. The format of the command line is the following:

command arg1 arg2 .. argn

I.e, if we want to execute some command the following statement can be used:

SET tokudb_dir_cmd = "command arg1 ... argn"

Currently the following commands are available:

• attach dictionary_name internal_file_name - attach internal_file_name to a dictionary_name, if the dictionary_name exists override the previous value, add new record otherwise
• detach dictionary_name - remove record with corresponding dictionary_name, the corresponding internal_file_name file stays untouched
• move old_dictionary_name new_dictionary_name - rename (only) dictionary_name from old_dictionary_name to new_dictionary_name

Information about the dictionary_name and internal_file_name can be found in the TokuDB_file_map table:

mysql> SELECT dictionary_name, internal_file_name FROM INFORMATION_SCHEMA.TokuDB_file_map;
+------------------------------+---------------------------------------------------------+
| dictionary_name              | internal_file_name                                      |
+------------------------------+---------------------------------------------------------+
| ./world/City-key-CountryCode | ./_world_sql_340a_39_key_CountryCode_12_1_1d_B_1.tokudb |
| ./world/City-main            | ./_world_sql_340a_39_main_12_1_1d_B_0.tokudb            |
| ./world/City-status          | ./_world_sql_340a_39_status_f_1_1d.tokudb               |
+------------------------------+---------------------------------------------------------+