SST Options¶

The following options specific to SST can be used in my.cnf under [sst].

option streamfmt¶

Values:	xbstream, tar
Default:	xbstream
Match:	Yes

Used to specify the Percona XtraBackup streaming format. The recommended value is streamfmt=xbstream. Certain features are not available with tar: encryption, compression, parallel streaming, streaming incremental backups. For more information about the xbstream format, see The xbstream Binary.

option transferfmt¶

Values:	socat, nc
Default:	socat
Match:	Yes

Used to specify the data transfer format. The recommended value is the default transferfmt=socat because it allows for socket options, such as transfer buffer sizes. For more information, see socat(1).

option tca¶

Example:	tca=~/etc/ssl/certs/mycert.crt

Used to specify the full path to the certificate authority (CA) file for socat encryption based on OpenSSL.

option tcert¶

Example:	tcert=~/etc/ssl/certs/mycert.pem

Used to specify the full path to the certificate file in PEM format for socat encryption based on OpenSSL.

option tkey¶

Used to specify the full path to the key file for SST encryption based on OpenSSL.

option encrypt¶

Values:	0, 1, 2, 3, 4
Default:	0
Match:	Yes

Used to enable and specify SST encryption mode:

• Set encrypt=0 to disable SST encryption. This is the default value.

• Set encrypt=1 to perform symmetric SST encryption based on XtraBackup.

  Note

  This mode has been deprecated.

• Set encrypt=2 to perform SST encryption based on OpenSSL with socat. Ensure that socat is built with OpenSSL: socat -V | grep OPENSSL. This is recommended if your nodes are over WAN and security constraints are higher.

  Note

  This mode has been deprecated.

• Set encrypt=3 to perform SST encryption based on SSL for just the key and certificate files as implemented in Galera.

  Note

  This mode has been deprecated.

  It does not provide certificate validation. In order to work correctly, paths to the key and certificate files need to be specified as well, for example:

  [sst]
  encrypt=3
  tkey=/etc/mysql/key.pem
  tcert=/etc/mysql/cert.pem
  

  Note

  The encrypt=3 option can only be used when wsrep_sst_method is set to xtrabackup-v2 (which is the default now).

• Set encrypt=4 for SST encryption with SSL files generated by MySQL. This is the recommended mode.

  Considering that you have all three necessary files:

  [sst]
  encrypt=4
  ssl-ca=ca.pem
  ssl-cert=server-cert.pem
  ssl-key=server-key.pem
  

For more information, see Encrypting PXC Traffic.

option encrypt-algo¶

Values:	AES128, AES192, AES256

Used to specify the SST encryption algorithm. It uses the same values as the --encryption option for Percona XtraBackup (see this document). The encrypt-algo option is considered only if encrypt is set to 1.

option sockopt¶

Used to specify key/value pairs of socket options, separated by commas, for example:

[sst]
sockopt="retry=2,interval=3"

The previous example causes socat to try to connect three times (initial attempt and two retries with a 3-second interval between attempts).

[sst]
sockopt=",cipher=AES128"

You can use the tcpwrap option to blacklist or whitelist clients. For more information about socket options, see socat (1).

option ncsockopt¶

Used to specify socket options for the netcat transfer format (transferfmt=nc).

option progress¶

Values:	1 or path to file

Used to specify where to write SST progress. If set to 1, it writes to MySQL stderr. Alternatively, you can specify the full path to a file. If this is a FIFO, it needs to exist and be open on reader end before itself, otherwise wsrep_sst_xtrabackup will block indefinitely.

option rebuild¶

Values:	0, 1
Default:	0

Used to enable rebuilding of index on joiner node. Set to 1 to enable. This is independent of compaction, though compaction enables it. Rebuild of indexes may be used as an optimization.

option time¶

Values:	0, 1
Default:	0

Enabling this option instruments key stages of backup and restore in SST.

option rlimit¶

Example:	rlimit=128k

Used to set a ratelimit in bytes. Add a suffix (k, m, g, t) to specify other units. For example, 128k is 128 kilobytes. Refer to pv(1) for details.

option incremental¶

Values:	0, 1
Default:	0

Used to supersede IST on joiner node. Requires manual setup and is not supported currently.

option use_extra¶

Values:	0, 1
Default:	0

Used to force SST to use the thread pool’s extra_port. Make sure that thread pool is enabled and the extra_port option is set in my.cnf before you enable this option.

option cpat¶

Used to define the files that need to be retained in the datadir before running SST, so that the state of the other node can be restored cleanly. For example:

[sst]
cpat='.*galera\.cache$\|.*sst_in_progress$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.xyz$'

option compressor¶

Default:	not set (disabled)
Example:	compressor=gzip

option decompressor¶

Default:	not set (disabled)
Example:	decompressor='gzip -dc'

Two previous options enable stream-based compression/decompression. When these options are set, compression/decompression is performed on stream, in contrast to earlier PXB-based one where decompression was done after streaming to disk, involving additional I/O. This saves a lot of I/O (up to twice less I/O on joiner node).

You can use any compression utility which works on stream: gzip, pigz (which is recommended because it is multi-threaded), etc. Compressor has to be set on donor node and decompressor on joiner node (although you can set them vice-versa for configuration homogeneity, it won’t affect that particular SST). To use XtraBackup based compression as before, set compress under [xtrabackup]. Having both enabled won’t cause any failure (although you will be wasting CPU cycles).

option inno-backup-opts¶

option inno-apply-opts¶

option inno-move-opts¶

Default:	Empty
Type:	Quoted String

This group of options can be used to pass innobackupex options for backup, apply, and move stages.

option sst-initial-timeout¶

Default:	100
Unit:	seconds

This option is used to configure initial timeout (in seconds) to receive the first packet via SST. This has been implemented, so that if the donor node fails somewhere in the process, the joiner node will not hang up and wait forever.

By default, the joiner node will not wait for more than 100 seconds to get a donor node. The default should be sufficient, however, it is configurable, so you can set it appropriately for your cluster. To disable initial SST timeout, set sst-initial-timeout=0.

option tmpdir¶

Version:	Introduced in 5.6.35-26.20-3
Default:	Empty
Example:	/path/to/tmp/dir

This option specifies the location for storing the temporary file where the transaction log is stored before streaming or copying it to a remote host.

The tmpdir option can be set in the following my.cnf groups:

• [sst] is the primary location (others are ignored)
• [xtrabackup] is the secondary location (if not specified under [sst])
• [mysqld] is used if it is not specified in either of the above

XtraBackup SST Dependencies¶

Each suppored version of Percona XtraDB Cluster is tested against a specific version of Percona XtraBackup:

• Percona XtraDB Cluster 5.6 requires Percona XtraBackup 2.3
• Percona XtraDB Cluster 5.7 requires Percona XtraBackup 2.4
• Percona XtraDB Cluster 8.0 requires Percona XtraBackup 8.0

Other combinations are not guaranteed to work.

The following are optional dependencies of Percona XtraDB Cluster introduced by wsrep_sst_xtrabackup (except for obvious and direct dependencies):

• qpress for decompression. It is an optional dependency of Percona XtraBackup and it is available in our software repositories.
• my_print_defaults to extract values from my.cnf. Provided by the server package.
• openbsd-netcat or socat for transfer. socat is a direct dependency of Percona XtraDB Cluster and it is the default.
• xbstream for streaming.
• pv is required for progress and rlimit.
• mkfifo is required for progress. Provided by coreutils.
• mktemp is required for incremental. Provided by coreutils.

XtraBackup-based Encryption¶

This is enabled when encrypt is set to 1 under [sst] in my.cnf. However, due to bug #1190335, it will also be enabled when you specify any of the following options under [xtrabackup] in my.cnf:

• encrypt
• encrypt-key
• encrypt-key-file

There is no way to disable encryption from innobackupex if any of the above are in my.cnf under [xtrabackup]. For that reason, consider the following scenarios:

1. If you want to use XtraBackup-based encryption for SST but not otherwise, use encrypt=1 under [sst] and provide the above Percona XtraBackup encryption options under [sst]. Details of those options can be found here.
2. If you want to use XtraBackup-based encryption always, use encrypt=1 under [sst] and have the above Percona XtraBackup encryption options either under [sst] or [xtrabackup].
3. If you don’t want to use XtraBackup-based encryption for SST, but want it otherwise, use encrypt=0 or encrypt=2 and do NOT provide any Percona XtraBackup encryption options under [xtrabackup]. You can still have them under [sst] though. You will need to provide those options on innobackupex command line then.
4. If you don’t want to use XtraBackup-based encryption at all (or only the OpenSSL-based for SST with encrypt=2), then don’t provide any Percona XtraBackup encryption options in my.cnf.

Memory Allocation¶

The amount of memory for XtraBackup is defined by the --use-memory option. You can pass it using the inno-apply-opts option under [sst] as follows:

[sst]
inno-apply-opts="--use-memory=500M"

If it is not specified, the use-memory option under [xtrabackup] will be used:

[xtrabackup]
use-memory=32M

If neither of the above are specified, the size of the InnoDB memory buffer will be used:

[mysqld]
innodb_buffer_pool_size=24M