Data at Rest Encryption

Data at Rest Encryption

Prerequisites

Data at rest encryption requires that a keyring plugin, such as keyring_file or Keyring Vault plugin be installed and already loaded. To load the keyring plugin when starting the server, use the --early-plugin-load option:

$ mysqld --early-plugin-load="keyring_file=keyring_file.so"

Altermatively, you can add this option to your configuration file:

[mysqld]
early-plugin-load=keyring_file.so

Warning

Only one keyring plugin should be enabled at a time. Enabling multiple keyring plugins is not supported and may result in data loss.

See also

MySQL Documentation:

Changing the Default Keyring Encryption

When encryption is enabled and the server is configured to use the KEYRING encryption, new tables use the default encryption key.

You many change this default encryption via the innodb_default_encryption_key_id variable.

See also

Configuring the way how tables are encrypted
innodb_encrypt_tables

System Variables

variable innodb_default_encryption_key_id
Version Info:
Command Line:

--innodb-default-encryption-key-id
Dynamic:

Yes
Scope:

Session
Variable Type:

Numeric
Default Value:

0

The ID of the default encryption key. By default, this variable contains 0 to encrypt new tables with the latest version of the key percona_innodb-0.

To change the default value use the following syntax:

mysql> SET innodb_default_encryption_key_id = NEW_ID

Here, NEW_ID is an unsigned 32-bit integer.

InnoDB System Tablespace Encryption

The InnoDB system tablespace is encrypted by using master key encryption. The server must be started with the --bootstrap option.

If the variable innodb_sys_tablespace_encrypt is set to ON and the server has been started in the bootstrap mode, you may create an encrypted table as follows:

mysql> CREATE TABLE ... TABLESPACE=innodb_system ENCRYPTION='Y'

Note

You cannot encrypt existing tables in the System tablespace.

It is not possible to convert the system tablespace from encrypted to unencrypted or vice versa. A new instance should be created and user tables must be transferred to the desired instance.

You can encrypt the already encrypted InnoDB system tablespace (key rotation) with a new master key by running the following ALTER INSTANCE statement:

mysql> ALTER INSTANCE ROTATE INNODB MASTER KEY

Doublewrite Buffers

The two types of doublewrite buffers used in Percona Server are encrypted differently.

When the InnoDB system tablespace is encrypted, the doublewrite buffer pages are encrypted as well. The key which was used to encrypt the InnoDB system tablespace is also used to encrypt the doublewrite buffer.

Percona Server encrypts the parallel doublewrite buffer with the respective tablespace keys. Only encrypted tablespace pages are written as encrypted in the parallel doublewrite buffer. Unencrypted tablespace pages will be written as unencrypted.

Important

A server instance bootstrapped with the encrypted InnoDB system tablespace cannot be downgraded. It is not possible to parse encrypted InnoDB system tablespace pages in a version of Percona Server lower than the version where the InnoDB system tablespace has been encrypted.

System variables

variable innodb_sys_tablespace_encrypt
Version Info:
Command Line:

--innodb-sys-tablespace-encrypt
Dynamic:

No
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

Enables the encryption of the InnoDB System tablespace. It is essential that the server is started with the --bootstrap option.

See also

MySQL Documentation: --bootstrap option
https://dev.mysql.com/doc/refman/5.7/en/server-options.html#option_mysqld_bootstrap
variable innodb_parallel_dblwr_encrypt
Version Info:
Command Line:

--innodb-parallel-dblwr-encrypt
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

Enables the encryption of the parallel doublewrite buffer. For encryption, uses the key of the tablespace where the parallel doublewrite buffer is used.

InnoDB General Tablespace Encryption

In Percona Server 5.7.20-18 existing tablespace encryption support has been extended to handle general tablespaces. A general tablespace is either fully encrypted, covering all the tables inside, or not encrypted at all. It is not possible to have encrypted only some of the tables in a general tablespace.

This feature extends the CREATE TABLESPACE statement to accept the ENCRYPTION='Y/N' option.

Usage

General tablespace encryption is enabled by the following syntax extension:

mysql> CREATE TABLESPACE tablespace_name ... ENCRYPTION='Y'

Attempts to create or to move tables, including partitioned ones, to a general tablespace with an incompatible encryption setting are diagnosed and aborted.

As you cannot move tables between encrypted and unencrypted tablespaces, you will need to create another table, add it to a specific tablespace and run INSERT INTO SELECT from the table you want to move from, and then you will get encrypted or decrypted table with your desired content.

Example

To create an encrypted tablespace run:

mysql> CREATE TABLESPACE foo ADD DATAFILE 'foo.ibd' ENCRYPTION='Y';

To add an encrypted table to that table space run:

mysql> CREATE TABLE t1 (a INT, b TEXT) TABLESPACE foo ENCRYPTION="Y";

Trying to add unencrypted table to this table space will result in an error:

mysql> CREATE TABLE t3 (a INT, b TEXT) TABLESPACE foo ENCRYPTION="N";
ERROR 1478 (HY000): InnoDB: Tablespace `foo` can contain only an ENCRYPTED tables.

Note

Percona XtraBackup currently doesn’t support backup of encrypted general tablespaces.

Checking

If there is a general tablespace which doesn’t include tables yet, sometimes user needs to find out whether it is encrypted or not (this task is easier for single tablespaces since you can check table info).

A flag field in the INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES has bit number 13 set if tablespace is encrypted. This bit can be ckecked with flag & 8192 expression in the following way:

>SELECT space, name, flag, (flag & 8192) != 0 AS encrypted FROM INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES WHERE name in ('foo', 'test/t2', 'bar', 'noencrypt');
+-------+-----------+-------+-----------+
| space | name      | flag  | encrypted |
+-------+-----------+-------+-----------+
|    29 | foo       | 10240 |      8192 |
|    30 | test/t2   |  8225 |      8192 |
|    31 | bar       | 10240 |      8192 |
|    32 | noencrypt |  2048 |         0 |
+-------+-----------+-------+-----------+
4 rows in set (0.01 sec)

System Variables

variable innodb_temp_tablespace_encrypt
Version Info:
Command Line:

--innodb-temp-tablespace-encrypt
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

Off

When this option is turned on, server starts to encrypt temporary tablespace and temporary InnoDB file-per-table tablespaces. The option does not force encryption of temporary tables which are currently opened, and it doesn’t rebuild system temporary tablespace to encrypt data which are already written.

Since temporary tablespace is created fresh at each server startup, it will not contain unencrypted data if this option specified as server argument.

Turning this option off at runtime makes server to create all subsequent temporary file-per-table tablespaces unencrypted, but does not turn off encryption of system temporary tablespace.

This feature is considered BETA quality.

Note

To use this option, keyring plugin must be loaded, otherwise server will give error message and refuse to create new temporary tables.

variable innodb_encrypt_tables
Version Info:
Command Line:

--innodb-encrypt-tables
Dynamic:

Yes
Scope:

Global
Variable Type:

Text
Default Value:

OFF

The implementation of the behavior controlled by this variable is considered BETA quality.

This variable was ported from MariaDB and then extended to support key rotation. This variable has the following possible values:

ON

New tables are created encrypted. You can create an unencrypted table by using the ENCRYPTION=NO clause to the CREATE TABLE or ALTER TABLE statement.

OFF

By default, newly created tables are not encrypted. Add the ENCRYPTION=YES clause in the CREATE TABLE or ALTER TABLE statement to create an encrypted table.

FORCE

New tables are created encrypted with the master key. Passing ENCRYPTION=NO to CREATE TABLE or ALTER TABLE will result in an error and the table will not be created or altered.

If you alter a table which was created without encryption, note that it will not be encrypted unless you use the ENCRYPTION clause explicitly.

KEYRING_ON

Availability:This value is Experimental quality

New tables are created encrypted with the keyring as the default encryption. You may specify a numeric key identifier and use a specific percona-innodb- key from the keyring instead of the default key:

mysql> CREATE TABLE ... ENCRYPTION=’KEYRING’ ENCRYPTION_KEY_ID=NEW_ID

NEW_ID is an unsigned 32-bit integer that refers to the numerical part of the percona_innodb- key. When you assign a numerical identifer in the ENCRYPTION_KEY_ID clause, the server uses the latest version of the corresponding key. For example, the clause ENCRYPTION_KEY_ID=2 refers to the latest version of the percona_innodb-2 key from the keyring.

In case the percona-innodb- key with the requested ID does not exist in the keyring, Percona Server will create it with version 1. If a new percona-innodb- key cannot be created with the requested ID, the whole CREATE TABLE statement fails

FORCE_KEYRING

Availability:This value is Experimental quality

New tables are created encrypted and keyring encryption is enforced.

ONLINE_TO_KEYRING

Availability:This value is Experimental quality

All tables created or altered without the ENCRYPTION=NO clause are encrypted with the latest version of the default encryption key. If a table being altered is already encrypted with the master key, the table is recreated encrypted with the latest version of the default encryption key.

ONLINE_TO_KEYRING_FORCE

Availability:This value is Experimental quality

It is only possible to apply the keyring encryption when creating or altering tables.

Note

The ALTER TABLE statement changes the current encryption mode only if you use the ENCRYPTION clause.

See also

MariaDB Documentation: innodb_encrypt_tables Option
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_encrypt_tables
variable innodb_online_encryption_threads
Version Info:
Command Line:

--innodb-online-encryption-threads
Dynamic:

Yes
Scope:

Global
Variable Type:

Numeric
Default Value:

1

This variable works in combination with the innodb_encrypt_tables variable set to ONLINE_TO_KEYRING. This variable configures the number of threads for background encryption. For the online encryption to work, this variable must contain a value greater than zero.

variable innodb_online_encryption_rotate_key_age
Version Info:
Command Line:

--innodb-online-encryption-rotate-key-age
Dynamic:

Yes
Scope:

Global
Variable Type:

Numeric
Default Value:

1

By using this variable, you can re-encrypt the table encrypted using KEYRING. The value of this variable determines how frequently the encrypted tables should be encrypted again. If it is set to 1, the encrypted table is re-encrypted on each key rotation. If it is set to 2, the table is encrypted on every other key rotation.

variable innodb_encrypt_online_alter_logs
Version Info:
Command Line:

--innodb-encrypt-online-alter-logs
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

This variable simultaneously turns on the encryption of files used by InnoDB for full text search using parallel sorting, building indexes using merge sort, and online DDL logs created by InnoDB for online DDL.

InnoDB Undo Tablespace Encryption

Availability:This feature is Experimental quality

The encryption of InnoDB Undo tablespaces is only available when using separate undo tablespaces. Otherwise, the InnoDB undo log is part of the InnoDB system tablespace.

See also

More information about how the encryption of the system tablespace
InnoDB System Tablespace Encryption

System variables

variable innodb_undo_log_encrypt
Version Info:
Command Line:

--innodb-undo-log-encrypt
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

Off

Enables the encryption of InnoDB Undo tablespaces. You can enable encryption and disable encryption while the server is running.

Note

If you enable undo log encryption, the server writes encryption information into the header. That information stays in the header during the life of the undo log. If you restart the server, the server will try to load the encryption key from the keyring during startup. If the keyring is not available, the server cannot start.

Binary log encryption

A new option, implemented since Percona Server 5.7.20-19, is encryption of binary and relay logs, triggered by the encrypt_binlog variable.

Besides turning encrypt_binlog ON, this feature requires both master_verify_checksum and binlog_checksum variables to be turned ON.

While replicating, master sends the stream of decrypted binary log events to a slave (SSL connections can be set up to encrypt them in transport). That said, masters and slaves use separate keyring storages and are free to use differing keyring plugins.

Dumping of encrypted binary logs involves decryption, and can be done using mysqlbinlog with --read-from-remote-server option.

Note

Taking into account that --read-from-remote-server option is only relevant to binary logs, encrypted relay logs can not be dumped/decrypted in this way.

Upgrading from Percona Server 5.7.20-19 to any higher version

The key format in the keyring vault plugin was changed for binlog encryption in Percona Server 5.7.20-19 release. When you are upgrading from Percona Server 5.7.20-19 to a higher version in the Percona Server 5.7 series or to a version prior to 8.0.15-5 in the Percona Server 8.0 series, the binary log encryption will work after you complete the following steps:

  1. Upgrade to a version higher than Percona Server 5.7.20-19
  2. Start the server without enabling the binary log encryption: :bash:`–encrypt_binlog=OFF`
  3. Enforce the key rotation: :mysql:`SELECT rotate_system_key(“percona_binlog”)`
  4. Restart the server enabling the binary log encryption: :bash:`–encrypt_binlog=ON`

See also

Percona Server Documentation: Important changes in Percona Server 8.0.15-5

System Variables

variable encrypt_binlog
Version Info:
Command Line:

--encrypt-binlog
Dynamic:

No
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

The variable turns on binary and relay logs encryption.

Redo Log Encryption

Availability:This feature is Experimental quality

InnoDB redo log encryption is enabled by setting the variable innodb_redo_log_encrypt. This variable has three values: MASTER_KEY, KEYRING_KEY and OFF (set by default).

MASTER_KEY uses the InnoDB master key to encrypt with unique keys for each log file in the redo log header.

KEYRING_KEY uses the percona_redo versioned key from the keyring. When innodb_redo_log_encrypt is set to KEYRING_KEY, each new redo log file is encrypted with the latest percona_redo key from the keyring.

System variables

variable innodb_redo_log_encrypt
Version Info:
Command Line:

--innodb-redo-log-encrypt
Dynamic:

Yes
Scope:

Global
Variable Type:

Text
Default Value:

OFF

Enables the encryption of the redo log.

variable innodb_scrub_log
Version Info:
Command Line:

--innodb-scrub-log
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

Specifies if data scrubbing should be automatically applied to the redo log.

variable innodb_scrub_log_speed
Version Info:
Command Line:

--innodb-scrub-log-speed
Dynamic:

Yes
Scope:

Global
Variable Type:

Text
Default Value:

Specifies the velocity of data scrubbing (writing dummy redo log records) in bytes per second.

Implemented in version 5.7.27-30, the key rotation is redesigned to allow SELECT rotate_system_key("percona_redo). The currently used key version is available in the innodb_redo_key_version status. The feature is Experimental.

Temporary file encryption

A new feature, implemented since Percona Server 5.7.22-22, is the encryption of temporary files, triggered by the encrypt-tmp-files option.

Temporary files are currently used in Percona Server for the following purposes:

This feature is considered Experimental quality.

  • filesort (for example, SELECT statements with SQL_BIG_RESULT hints),
  • binary log transactional caches,
  • Group Replication caches.

For each temporary file, an encryption key is generated locally, only kept in memory for the lifetime of the temporary file, and discarded afterwards.

System Variables

variable encrypt-tmp-files
Version Info:
Command Line:

--encrypt-tmp-files
Dynamic:

No
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

The option turns on encryption of temporary files created by Percona Server.

Key Rotation

The keyring management is enabled for each tablespace separately when you set the encryption in the ENCRYPTION clause, to KEYRING in the supported SQL statement:

  • CREATE TABLE .. ENCRYPTION=’KEYRING`
  • ALTER TABLE ... ENCRYPTION=’KEYRING’
  • CREATE TABLESPACE tablespace_name … ENCRYPTION=’KEYRING’

Note

Running ALTER TABLE .. ENCRYPTION=’Y’ on the tablespace created with ENCRYPTION=’KEYRING’ converts the table back to the existing MySQL scheme.

Keyring Vault plugin

In Percona Server 5.7.20-18 a keyring_vault plugin has been implemented that can be used to store the encryption keys inside the Hashicorp Vault server.

Important

keyring_vault plugin only works with kv secrets engine version 1.

See also

HashiCorp Documentation: More information about kv secrets engine
https://www.vaultproject.io/docs/secrets/kv/kv-v1.html

Installation

The safest way to load the plugin is to do it on the server startup by using –early-plugin-load option option:

$ mysqld --early-plugin-load="keyring_vault=keyring_vault.so" \
--loose-keyring_vault_config="/home/mysql/keyring_vault.conf"

It should be loaded this way to be able to facilitate recovery for encrypted tables.

Warning

If server should be started with several plugins loaded early, --early-plugin-load should contain their list separated by semicolons. Also it’s a good practice to put this list in double quotes so that semicolons do not create problems when executed in a script.

Apart from installing the plugin you also need to set the keyring_vault_config variable. This variable should point to the keyring_vault configuration file, whose contents are discussed below.

This plugin supports the SQL interface for keyring key management described in General-Purpose Keyring Key-Management Functions manual.

To enable the functions you’ll need to install the keyring_udf plugin:

mysql> INSTALL PLUGIN keyring_udf SONAME 'keyring_udf.so';

Usage

On plugin initialization keyring_vault connects to the Vault server using credentials stored in the credentials file. Location of this file is specified in by keyring_vault_config. On successful initialization it retrieves keys signatures and stores them inside an in-memory hash map.

Configuration file should contain the following information:

  • vault_url - the address of the server where Vault is running. It can be a named address, like one in the following example, or just an IP address. The important part is that it should begin with https://.
  • secret_mount_point - the name of the mount point where keyring_vault will store keys.
  • token - a token generated by the Vault server, which keyring_vault will further use when connecting to the Vault. At minimum, this token should be allowed to store new keys in a secret mount point (when keyring_vault is used only for transparent data encryption, and not for keyring_udf plugin). If keyring_udf plugin is combined with keyring_vault, this token should be also allowed to remove keys from the Vault (for the keyring_key_remove operation supported by the keyring_udf plugin).
  • vault_ca [optional] - this variable needs to be specified only when the Vault’s CA certificate is not trusted by the machine that is going to connect to the Vault server. In this case this variable should point to CA certificate that was used to sign Vault’s certificates.

Warning

Each secret_mount_point should be used by only one server - otherwise mixing encryption keys from different servers may lead to undefined behavior.

An example of the configuration file looks like this:

vault_url = https://vault.public.com:8202
secret_mount_point = secret
token = 58a20c08-8001-fd5f-5192-7498a48eaf20
vault_ca = /data/keyring_vault_confs/vault_ca.crt

When a key is fetched from a keyring for the first time the keyring_vault communicates with the Vault server, and retrieves the key type and data. Next it queries the Vault server for the key type and data and caches it locally.

Key deletion will permanently delete key from the in-memory hash map and the Vault server.

Note

Percona XtraBackup currently doesn’t support backup of tables encrypted with Keyring Vault plugin.

System Variables

variable keyring_vault_config
Version Info:
Command Line:

--keyring-vault-config
Dynamic:

Yes
Scope:

Global
Variable Type:

Text
Default Value:

This variable is used to define the location of the Keyring Vault plugin configuration file.

variable keyring_vault_timeout
Version Info:
Command Line:

--keyring-vault-timeout
Dynamic:

Yes
Scope:

Global
Variable Type:

Numeric
Default Value:

15

This variable allows to set the duration in seconds for the Vault server connection timeout. Default value is 15. Allowed range is from 1 second to 86400 seconds (24 hours). The timeout can be also completely disabled to wait infinite amount of time by setting this variable to 0.

Data Scrubbing

While data encryption ensures that the existing data are not stored in plain form, the data scrubbing literally removes the data once the user decides they should be deleted. Compare this behavior with how the DELETE statement works which only marks the affected data as deleted - the space claimed by this data is overwritten with new data later.

Once enabled, data scrubbing works automatically on each tablespace separately. To enable data scrubbing, you need to set the following variables:

  • innodb-background-scrub-data-uncompressed
  • innodb-background-scrub-data-compressed

Uncompressed tables can also be scrubbed immediately, independently of key rotation or background threads. This can be enabled by setting the variable innodb-immediate-scrub-data-uncompressed. This option is not supported for compressed tables.

Note that data scrubbing is made effective by setting the innodb_online_encryption_threads variable to a value greater than zero.

System Variables

variable innodb_background_scrub_data_compressed
Version Info:
Command Line:

--innodb-background-scrub-data-compressed
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF
variable innodb_background_scrub_data_uncompressed
Version Info:
Command Line:

--innodb-background-scrub-data-uncompressed
Dynamic:

Yes
Scope:

Global
Variable Type:

Boolean
Default Value:

OFF

See also

Vault Documentation
https://www.vaultproject.io/docs/index.html
MySQL Documentation: General-Purpose Keyring Key-Management Functions
https://dev.mysql.com/doc/refman/5.7/en/keyring-udfs-general-purpose.html
Download PDF Manual

Table Of Contents

This Page

Percona Server Series