Buy Percona ServicesBuy Now!

Encrypting PXC Traffic

Percona XtraDB Cluster supports encryption for all traffic involved in cluster operation. This includes traffic between client applications and cluster nodes, SST, IST, write-set replication, and various service messages.

Generating Keys and Certificates

To encrypt Percona XtraDB Cluster traffic, you must generate the following sets of files:

  • Certificate Authority (CA) key and certificate to sign the server and client certificates.
  • Server key and certificate to secure database server activity and write-set replication traffic.
  • Client key and certificate to secure client communication traffic.

These files should be generated using OpenSSL.

Note

The Common Name value used for the server and client keys and certificates must differ from that value used for the CA certificate.

Generating CA Key and Certificate

The Certificate Authority is used to verify the signature on certificates.

  1. Generate the CA key file:

    $ openssl genrsa 2048 > ca-key.pem
    
  2. Generate the CA certificate file:

    $ openssl req -new -x509 -nodes -days 3600
        -key ca-key.pem -out ca.pem
    

Generating Server Key and Certificate

  1. Generate the server key file:

    $ openssl req -newkey rsa:2048 -days 3600 \
        -nodes -keyout server-key.pem -out server-req.pem
    
  2. Remove the passphrase:

    $ openssl rsa -in server-key.pem -out server-key.pem
    
  3. Generate the server certificate file:

    $ openssl x509 -req -in server-req.pem -days 3600 \
        -CA ca.pem -CAkey ca-key.pem -set_serial 01 \
        -out server-cert.pem
    

Generating Client Key and Certificate

  1. Generate the client key file:

    $ openssl req -newkey rsa:2048 -days 3600 \
        -nodes -keyout client-key.pem -out client-req.pem
    
  2. Remove the passphrase:

    $ openssl rsa -in client-key.pem -out client-key.pem
    
  3. Generate the client certificate file:

    $ openssl x509 -req -in client-req.pem -days 3600 \
        -CA ca.pem -CAkey ca-key.pem -set_serial 01 \
        -out client-cert.pem
    

Verifying Certificates

To verify that the server and client certificates are correctly signed by the CA certificate, run the following command:

$ openssl verify -CAfile ca.pem server-cert.pem client-cert.pem

If the verification is successful, you should see the following output:

server-cert.pem: OK
client-cert.pem: OK

Deploying Keys and Certificates

Use a secure method (for example, scp or sftp) to send the key and certificate files to each node. Place them under the /etc/mysql/certs/ directory or similar location where you can find them later.

Note

Make sure that this directory is protected with proper permissions. Most likely, you only want to give read permissions to the user running mysqld.

The following files are required:

  • Certificate Authority certificate file (ca.pem)

    This file is used to verify signatures.

  • Server key and certificate files (server-key.pem and server-cert.pem)

    These files are used to secure database server activity and write-set replication traffic.

  • Client key and certificate files (client-key.pem and client-cert.pem)

    These files are required only if the node should act as a MySQL client. For example, if you are planning to perform SST using mysqldump.

Enabling Encryption

To enable encryption, you need to specify the location of the required key and certificate files in the Percona XtraDB Cluster configuration. If you do not have the necessary files, see Generating Keys and Certificates.

Note

Encryption settings are not dynamic. To enable it on a running cluster, you need to restart the entire cluster.

There are three aspects of Percona XtraDB Cluster operation, where you can enable encryption:

Encrypting Database Traffic

Percona XtraDB Cluster uses the underlying MySQL encryption mechanism to secure communication between client applications and cluster nodes. Specify the following settings in the my.cnf configuration file for each node:

[mysqld]
ssl-ca=/etc/mysql/certs/ca.pem
ssl-cert=/etc/mysql/certs/server-cert.pem
ssl-key=/etc/mysql/certs/server-key.pem

[client]
ssl-ca=/etc/mysql/certs/ca.pem
ssl-cert=/etc/mysql/certs/client-cert.pem
ssl-key=/etc/mysql/certs/client-key.pem

After you restart the node, it will use these files for encrypting communication with clients. MySQL clients require only the second part of the configuration to communicate with cluster nodes.

Encrypting Replication Traffic

Replication traffic refers to the following:

  • Write-set replication is the main workload of Percona XtraDB Cluster (replicating transactions that execute on one node to all other nodes).
  • Incremental State Transfer (IST) is copying only missing transactions from DONOR to JOINER node.
  • Service messages ensure that all nodes are synchronized.

All of this traffic is transferred via the same underlying communication channel (gcomm). Securing this channel will ensure that IST traffic, write-set replication, and service messages are encypted.

To enable encryption for all these processes, define the paths to the key, certificate and certificate authority files using the following wsrep provider options:

To set these options, use the wsrep_provider_options variable in the configuration file:

wsrep_provider_options="socket.ssl=yes;socket.ssl_ca=/etc/mysql/certs/ca.pem;socket.ssl_cert=/etc/mysql/certs/server-cert.pem;socket.ssl_key=/etc/mysql/certs/server-key.pem"

Note

You must use the same key and certificate files on all nodes, preferably those used for Encrypting Database Traffic.

Upgrading Certificates

The following procedure shows how to upgrade certificates used for securing replication traffic when there are two nodes in the cluster:

  1. Restart the first node with the socket.ssl_ca option set to a combination of the the old and new certificates in a single file.

    For example, you can merge contents of old-ca.pem and new-ca.pem into upgrade-ca.pem as follows:

    cat old-ca.pem > upgrade-ca.pem && \
    cat new-ca.pem >> upgrade-ca.pem
    

    Set the wsrep_provider_options variable as follows:

    wsrep_provider_options="socket.ssl=yes;socket.ssl_ca=/etc/mysql/certs/upgrade-ca.pem;socket.ssl_cert=/etc/mysql/certs/old-cert.pem;socket.ssl_key=/etc/mysql/certs/old-key.pem"
    
  2. Restart the second node with the socket.ssl_ca, socket.ssl_cert, and socket.ssl_key options set to the corresponding new certificate files.

    wsrep_provider_options="socket.ssl=yes;socket.ssl_ca=/etc/mysql/certs/new-ca.pem;socket.ssl_cert=/etc/mysql/certs/new-cert.pem;socket.ssl_key=/etc/mysql/certs/new-key.pem"
    
  3. Restart the first node with the new certificate files, as in the previous step.

  4. You can remove the old certificate files.

Encrypting SST Traffic

This refers to full data transfer that usually occurs when a new node (JOINER) joins the cluster and receives data from an existing node (DONOR).

For more information, see State Snapshot Transfer.

When copying encrypted data via SST, the keyring must be sent over with the files for decryption. Make sure that the following options are set in my.cnf on all nodes:

early-plugin-load=keyring_file.so
keyring-file-data=/path/to/keyring/file

Warning

The cluster will not work if keyring configuration across nodes is different.

The following SST methods are available: rsync, mysqldump, and xtrabackup.

rsync

This SST method does not support encryption. Avoid using this method if you need to secure traffic between DONOR and JOINER nodes.

mysqldump

This SST method dumps data from DONOR and imports it to JOINER. Encryption in this case is performed using the same certificates configured for Encrypting Database Traffic, because mysqldump connects through the database client.

Here is how to enable encryption for SST using mysqldump in a running cluster:

  1. Create a user for SST on one of the nodes:

    mysql> CREATE USER 'sst_user'$'%' IDENTIFIED BY PASSWORD 'sst_password';
    

    Note

    This user must have the same name and password on all nodes where you want to use mysqldump for SST.

  2. Grant usage privileges to this user and require SSL:

    mysql> GRANT USAGE ON *.* TO 'sst_user' REQUIRE SSL;
    
  3. To make sure that the SST user replicated across the cluster, run the following query on another node:

    mysql> SELECT User, Host, ssl_type FROM mysql.user WHERE User='sst_user';
    
    +----------+------+----------+
    | User     | Host | ssl_type |
    +----------+------+----------+
    | sst_user | %    | Any      |
    +----------+------+----------+
    

    Note

    If the wsrep_OSU_method is set to ROI, you need to manually create the SST user on each node in the cluster.

  4. Specify corresponding certificate files in both [mysqld] and [client] sections of the configuration file on each node:

    [mysqld]
    ssl-ca=/etc/mysql/certs/ca.pem
    ssl-cert=/etc/mysql/certs/server-cert.pem
    ssl-key=/etc/mysql/certs/server-key.pem
    
    [client]
    ssl-ca=/etc/mysql/certs/ca.pem
    ssl-cert=/etc/mysql/certs/client-cert.pem
    ssl-key=/etc/mysql/certs/client-key.pem
    

    For more information, see Encrypting Database Traffic.

  5. Also specify the SST user credentials in the wsrep_sst_auth variable on each node:

    [mysqld]
    wsrep_sst_auth = sst_user:sst_password
    
  6. Restart the cluster with the new configuration.

If you do everything correctly, mysqldump will connect to DONOR with the SST user, generate a dump file, and import it to JOINER node.

xtrabackup

This is the default SST method (the wsrep_sst_method is set to xtrabackup-v2), which uses Percona XtraBackup to perform non-blocking transfer of files. For more information, see Percona XtraBackup SST Configuration.

Encryption mode for this method is selected using the encrypt option:

  • encrypt=0 is the default value, meaning that encryption is disabled.

  • encrypt=1, encrypt=2, and encrypt=3 have been deprecated.

  • encrypt=4 enables encryption based on key and certificate files generated with OpenSSL. For more informations, see Generating Keys and Certificates.

    To enable encryption for SST using XtraBackup, specify the location of the keys and certificate files in the each node’s configuration under [sst]:

    [sst]
    encrypt=4
    ssl-ca=/etc/mysql/certs/ca.pem
    ssl-cert=/etc/mysql/certs/server-cert.pem
    ssl-key=/etc/mysql/certs/server-key.pem
    

Note

SSL clients require DH parameters to be at least 1024 bits, due to the logjam vulnearability. However, versions of socat earlier than 1.7.3 use 512-bit parameters. If a dhparams.pem file of required length is not found during SST in the data directory, it is generated with 2048 bits, which can take several minutes. To avoid this delay, create the dhparams.pem file manually and place it in the data directory before joining the node to the cluster:

openssl dhparam -out /path/to/datadir/dhparams.pem 2048

For more information, see this blog post.

SSL Automatic Configuration

Percona XtraDB Cluster includes the pxc-encrypt-cluster-traffic variable that enables automatic configuration of SSL encrytion. By default, it is disabled, meaning that you need to configure SSL manually if you want to encrypt SST and IST traffic, and internal communication.

This variable cannot be changed on the command line during runtime. To enable automatic configuration of SSL encryption, set pxc-encrypt-cluster-traffic=ON in the my.cnf file and restart the cluster.

When you enable automatic configuration of SSL encryption, it looks for necessary key and certificate files in the ssl-ca, ssl-cert, and ssl-key options under [mysqld]. If these options are not set, it then looks in the data directory for ca.pem, server-cert.pem, and server-key.pem files.

Note

The [sst] section is not searched.

If all three files are found, they are used to configure encryption for SST using XtraBackup with encrypt=4. Other modes are deprecated and will be overriden if automatic configuration of SSL encryption is enabled. If any of the files are missing, a fatal error is generated.

The following settings are applied (and overridden if necessary):

[mysqld]
wsrep_provider_options=”socket.ssl_key=server-key.pem;socket.ssl_cert=server-cert.pem;socket.ssl_ca=ca.pem”

[sst]
encrypt=4
ssl-key=server-key.pem
ssl-ca=ca.pem
ssl-cert=server-cert.pem

For wsrep_provider_options, only the mentioned options are affected (socket.ssl_key, socket,ssl_cert, and socket.ssl_ca), the rest are not modified.

Visit Percona Store


General Inquiries

For general inquiries, please send us your question and someone will contact you.