KMIP (Key Management Interoperability Protocol) is an open standard developed by OASIS (Organization for Advancement of Structured Information Standards) for the encryption of stored data and cryptographic key management.
Percona Server for MySQL 8.0.27 and Percona XtraBackup 8.0.27 now include a KMIP keyring plugin to enable the exchange of cryptographic keys between a key management server and the database for encryption purposes. The procedure to use them with HashiCorp Vault Enterprise is described below.
Install Hashicorp Vault Enterprise
We will first install Hashicorp Vault Enterprise on Ubuntu Linux “Bionic” and then enable the KMIP secrets engine. The KMIP secrets engine is only available with the Enterprise version of HashiCorp Vault, hence a valid license for it is required.
Add HashiCorp repository and install enterprise vault package:
|
1 2 3 |
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add - sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" sudo apt-get update && sudo apt-get install vault-enterprise |
Export the license as an environment variable:
|
1 |
export VAULT_LICENSE=XXXX |
Create a configuration file to be used with the vault, vault_config.hcl:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
disable_mlock = true default_lease_ttl = "24h" max_lease_ttl = "24h" storage "file" { path = "/home/manish.chawla/vault/data" } listener "tcp" { address = "127.0.0.1:8200" tls_cert_file = "/home/manish.chawla/test_mode/certificates/vault.crt" tls_key_file = "/home/manish.chawla/test_mode/certificates/vault.key" } |
Note: Vault root certificates and key need to be created separately and are not covered here.
Start vault server with the configuration file:
|
1 |
vault server -config=$HOME/vault_config.hcl 2>&1 & |
Note: To configure and start the vault using systemd, refer to the instructions here.
Initialize the vault:
|
1 |
vault operator init -address=https://127.0.0.1:8200 |
This will generate five unseal keys and the initial root token.
|
1 2 3 4 5 6 7 |
Unseal Key 1: rf4C6gY87tN0UkJbJY96Aq4+Zext1YqgwaDnm+0gBBAH Unseal Key 2: f9KA7EdlF391cIUzDqaS1N21JjML/36sZFl1x/OfCPn4 Unseal Key 3: 8Nh3EFYF7S0S9qqzdUIilPPRZRmWaDG+3El4rr4FmZNX Unseal Key 4: iJaCKBIzxulLvA/6vbF3fRK1RXVZ0zLEZoVdlv/s13Sc Unseal Key 5: rA4pwT6EZLwmVXPQJfU9fjgeGwPaHl260qM9CVNiUw13 Initial Root Token: s.WXEZ26Yb3MtvzbvNMMIG8bve |
Unseal the vault.
Use any three unseal keys to unseal the vault. Three keys are required to unseal the vault.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
vault operator unseal -address=https://127.0.0.1:8200 rf4C6gY87tN0UkJbJY96Aq4+Zext1YqgwaDnm+0gBBAH Key Value --- ----- Seal Type shamir Initialized true Sealed true Total Shares 5 Threshold 3 Unseal Progress 1/3 Unseal Nonce aa1a25c8-aa90-49b3-b127-875524de38f8 Version 1.9.3+ent Storage Type file HA Enabled false vault operator unseal -address=https://127.0.0.1:8200 f9KA7EdlF391cIUzDqaS1N21JjML/36sZFl1x/OfCPn4 Key Value --- ----- Seal Type shamir Initialized true Sealed true Total Shares 5 Threshold 3 Unseal Progress 2/3 Unseal Nonce aa1a25c8-aa90-49b3-b127-875524de38f8 Version 1.9.3+ent Storage Type file HA Enabled false vault operator unseal -address=https://127.0.0.1:8200 8Nh3EFYF7S0S9qqzdUIilPPRZRmWaDG+3El4rr4FmZNX Key Value --- ----- Seal Type shamir Initialized true Sealed false Total Shares 5 Threshold 3 Version 1.9.3+ent Storage Type file Cluster Name vault-cluster-7d9b43de Cluster ID c047dcb5-5038-5a29-f4af-47c1ad560f9c HA Enabled false |
The vault is unsealed.
To use the vault in any terminal, run:
|
1 2 3 |
export VAULT_ADDR=https://127.0.0.1:8200 export VAULT_TOKEN=s.WXEZ26Yb3MtvzbvNMMIG8bve export VAULT_CACERT=/home/manish.chawla/test_mode/certificates/root.cer |
Configure KMIP Secrets Engine in Vault
Enable KMIP secrets engine:
|
1 2 |
vault secrets enable kmip Success! Enabled the kmip secrets engine at: kmip/ |
View the secrets list:
|
1 2 3 4 5 6 7 |
vault secrets list Path Type Accessor Description ---- ---- -------- ----------- cubbyhole/ cubbyhole cubbyhole_ec12856f per-token private secret storage identity/ identity identity_12d9670d identity store kmip/ kmip kmip_5fb3d4c6 n/a sys/ system system_1733eece system endpoints used for control, policy and debugging |
Change the kmip server listening address and port:
|
1 2 |
vault write kmip/config listen_addrs=0.0.0.0:5696 Success! Data written to: kmip/config |
Note: Here kmip is the default path of the secret engine and not the type of the engine.
By default, the kmip generates certificates in EC(Elliptic Curve). We need RSA for MySQL, so specify the certificate type (tls_ca_key_type) and bits (tls_ca_key_bits) to configure the kmip server.
|
1 2 |
vault write kmip/config tls_ca_key_type="rsa" tls_ca_key_bits=2048 Success! Data written to: kmip/config |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
vault read kmip/config Key Value --- ----- default_tls_client_key_bits 256 default_tls_client_key_type ec default_tls_client_ttl 336h listen_addrs [0.0.0.0:5696] server_hostnames [localhost] server_ips [127.0.0.1 ::1] tls_ca_key_bits 2048 tls_ca_key_type rsa tls_min_version tls12 |
The KMIP secrets engine uses scopes to partition object storage into multiple named buckets. Within a scope, roles can be created with a set of allowed operations that the particular role can perform.
Create a scope:
|
1 2 |
vault write -f kmip/scope/my-service Success! Data written to: kmip/scope/my-service |
Create a role within the scope, specifying the set of operations to allow or deny.
|
1 2 |
vault write kmip/scope/my-service/role/admin operation_all=true Success! Data written to: kmip/scope/my-service/role/admin |
Client Certificate Generation for the scope and role created above.
Retrieve the generated CA certificate:
|
1 |
vault read kmip/ca |
Copy and save the CA certificate as ca.pem.
Generate a certificate in PEM format, and save it in a JSON file named credential.json.
|
1 2 3 |
vault write -format=json \ kmip/scope/my-service/role/admin/credential/generate \ format=pem > credential.json |
Extract the certificate from the credential.json using jq tool and save it in a file named cert.pem.
|
1 |
jq -r .data.certificate < credential.json > cert.pem |
Extract the private key from the credential.json using jq tool and save it in a file named key.pem.
|
1 |
jq -r .data.private_key < credential.json > key.pem |
The KMIP configuration is now complete.
Percona Server for MySQL 8.0.27 Configuration for KMIP
This section describes the KMIP configuration in Percona Server for MySQL. KMIP is configured as a component in Percona Server for MySQL.
Create the global manifest file(mysqld.my) in the mysqld installation directory.
|
1 2 3 |
{ "components": "file://component_keyring_kmip" } |
Create the global configuration file, component_keyring_kmip.cnf in the directory, where the component_keyring_kmip library resides.
|
1 |
{ "path": "/home/manish.chawla/keyring_kmip", "server_addr": "0.0.0.0", "server_port": "5696", "client_ca": "/home/manish.chawla/cert.pem", "client_key": "/home/manish.chawla/key.pem", "server_ca": "/home/manish.chawla/ca.pem" } |
Note: SElinux/AppArmor rules may have to be adjusted, so that Percona Server for MySQL and Percona XtraBackup can access the certificates.
Initialize and start mysqld with encryption options(add in my.cnf):
|
1 |
--innodb-undo-log-encrypt --innodb-redo-log-encrypt --binlog-encryption --default-table-encryption=ON --log-replica-updates --gtid-mode=ON --enforce-gtid-consistency --binlog-format=row --source-verify-checksum=ON --binlog-checksum=CRC32 --table-encryption-privilege-check=ON |
Check the KMIP component status:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
8.0.27>SELECT * FROM performance_schema.keyring_component_status; +---------------------+------------------------------+ | STATUS_KEY | STATUS_VALUE | +---------------------+------------------------------+ | Component_name | component_keyring_kmip | | Author | Percona Corporation | | License | GPL | | Implementation_name | component_keyring_kmip | | Version | 1.0 | | Component_status | Active | | Server_addr | 0.0.0.0 | | Server_port | 5696 | | Client_ca | /home/manish.chawla/cert.pem | | Client_key | /home/manish.chawla/key.pem | | Server_ca | /home/manish.chawla/ca.pem | | Object_group | <NONE> | +---------------------+------------------------------+ |
Create some encrypted tables and add data in the Percona Server for MySQL.
Backup and Restore of Percona Server for MySQL 8.0.27 Using Percona XtraBackup 8.0.27
This section describes the procedure for taking backup and restore of Percona Server for MySQL 8.0.27 when the KMIP component is enabled and the KMIP vault server is running. Percona XtraBackup reads the KMIP configuration in Percona Server for MySQL automatically, and it is not required to pass this information separately.
Take full backup:
|
1 |
xtrabackup --user=backup --password=* --backup --target-dir=backup_directory |
Prepare full backup:
|
1 |
xtrabackup --prepare --target_dir=backup_directory |
Stop Percona Server for MySQL and move the data directory to another location. Disable SElinux/AppArmor before restoring the backup.
Restore full backup:
|
1 |
xtrabackup --copy-back --target-dir=backup_directory |
Change the ownership of the copied files in the Percona Server for MySQL data directory to the MySQL user.
Start Percona Server for MySQL and check the data. Enable SElinux/AppArmor, if disabled previously.
About the Author
Manish ChawlaManish joined Percona in 2018 and works in the QA team. Having worked in different roles, domains, and companies, he now tests Percona Server for MySQL, Percona XtraBackup, and Percona XtraDB Cluster. He likes testing complex distributed systems and has good experience in test planning and strategy.
