Making scheduled backups¶

Since backups are stored separately on the Amazon S3, a secret with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY should be present on the Kubernetes cluster. The secrets file with these base64-encoded keys should be created: for example deploy/backup-s3.yaml file with the following contents:

apiVersion: v1
kind: Secret
metadata:
  name: my-cluster-name-backup-s3
type: Opaque
data:
  AWS_ACCESS_KEY_ID: UkVQTEFDRS1XSVRILUFXUy1BQ0NFU1MtS0VZ
  AWS_SECRET_ACCESS_KEY: UkVQTEFDRS1XSVRILUFXUy1TRUNSRVQtS0VZ

The name value is the Kubernetes secret name which will be used further, and AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are the keys to access S3 storage (and obviously they should contain proper values to make this access possible). To have effect secrets file should be applied with the appropriate command to create the secret object, e.g. kubectl apply -f deploy/backup-s3.yaml (for Kubernetes).

Backups schedule is defined in the backup section of the deploy/cr.yaml file. This section contains following subsections:

• storages subsection contains data needed to access the S3-compatible cloud to store backups.
• schedule subsection allows to actually schedule backups (the schedule is specified in crontab format).

Here is an example of deploy/cr.yaml which uses Amazon S3 storage for backups:

...
backup:
  ...
  storages:
    s3-us-west:
      type: s3
      s3:
        bucket: S3-BACKUP-BUCKET-NAME-HERE
        region: us-west-2
        credentialsSecret: my-cluster-name-backup-s3
  ...
  schedule:
   - name: "sat-night-backup"
     schedule: "0 0 * * 6"
     storageName: s3-us-west
  ...

if you use some S3-compatible storage instead of the original Amazon S3, the endpointUrl is needed in the s3 subsection which points to the actual cloud used for backups and is specific to the cloud provider. For example, using Google Cloud involves the following endpointUrl:

endpointUrl: https://storage.googleapis.com

The options within these three subsections are further explained in the Custom Resource options.

One option which should be mentioned separately is credentialsSecret which is a Kubernetes secret for backups. Value of this key should be the same as the name used to create the secret object (my-cluster-name-backup-s3 in the last example).

The schedule is specified in crontab format as explained in Custom Resource options.

Making on-demand backup¶

To make an on-demand backup, the user should first configure the backup storage in the backup.storages subsection of the deploy/cr.yaml configuration file in a same way it was done for scheduled backups. When the deploy/cr.yaml file contains correctly configured storage and is applied with kubectl command, use a special backup configuration YAML file with the following contents:

• backup name in the metadata.name key,
• Percona XtraDB Cluster name in the spec.pxcCluster key,
• storage name from deploy/cr.yaml in the spec.storageName key.

The example of the backup configuration file is deploy/backup/backup.yaml.

When the backup destination is configured and applied with kubectl apply -f deploy/cr.yaml command, the actual backup command is executed:

kubectl apply -f deploy/backup/backup.yaml

cat <<EOF | kubectl apply -f-
apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterBackup
metadata:
  name: backup1
spec:
  pxcCluster: cluster1
  storageName: s3-us-west
EOF

Storing binary logs for point-in-time recovery¶

Point-in-time recovery functionality allows users to roll back the cluster to a specific transaction, time (or even skip a transaction in some cases). Technically, this feature involves continuously saving binary log updates to the backup storage. Point-in-time recovery is off by default.

To be used, it requires setting a number of keys in the pitr subsection under the backup section of the deploy/cr.yaml file:

• enabled key should be set to true,
• storageName key should point to the name of the storage already configured in the storages subsection (currently, only s3-compatible storages are supported),
• timeBetweenUploads key specifies the number of seconds between running the binlog uploader.

Following example shows how the pitr subsection looks like:

backup:
  ...
  pitr:
    enabled: true
    storageName: s3-us-west
    timeBetweenUploads: 60

Storing backup on Persistent Volume¶

Here is an example of the deploy/cr.yaml backup section fragment, which configures a private volume for filesystem-type storage:

...
backup:
  ...
  storages:
    fs-pvc:
      type: filesystem
      volume:
        persistentVolumeClaim:
          accessModes: [ "ReadWriteOnce" ]
          resources:
            requests:
              storage: 6Gi
  ...

Enabling compression for backups¶

There is a possibility to enable LZ4 compression for backups.

To enable compression, use pxc.configuration key in the deploy/cr.yaml configuration file to supply Percona XtraDB Cluster nodes with two additional my.cnf options under its [sst] and [xtrabackup] sections as follows:

pxc:
  image: percona/percona-xtradb-cluster:8.0.19-10.1
  configuration: |
    ...
    [sst]
    xbstream-opts=--decompress
    [xtrabackup]
    compress=lz4
    ...

When enabled, compression will be used for both backups and SST.

Restore the cluster from a previously saved backup¶

Backup can be restored not only on the Kubernetes cluster where it was made, but also on any Kubernetes-based environment with the installed Operator.

Following things are needed to restore a previously saved backup:

• Make sure that the cluster is running.

• Find out correct names for the backup and the cluster. Available backups can be listed with the following command:

  kubectl get pxc-backup
  

  Note

  Obviously, you can make this check only on the same cluster on which you have previously made the backup.

  And the following command will list existing Percona XtraDB Cluster names in the current Kubernetes-based environment:

  kubectl get pxc
  

cat <<EOF | kubectl apply -f-
apiVersion: "pxc.percona.com/v1"
kind: "PerconaXtraDBClusterRestore"
metadata:
  name: "restore1"
spec:
  pxcCluster: "cluster1"
  backupName: "backup1"
EOF

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: restore1
spec:
  pxcCluster: cluster1
  backupName: backup1
  pitr:
    type: date
    date: "2020-12-31 09:37:13"
    backupSource:
      storageName: "s3-us-west"

cat <<EOF | kubectl apply -f-
apiVersion: "pxc.percona.com/v1"
kind: "PerconaXtraDBClusterRestore"
metadata:
  name: "restore1"
spec:
  pxcCluster: "cluster1"
  backupName: "backup1"
  pitr:
    type: date
    date: "2020-12-31 09:37:13"
    backupSource:
      storageName: "s3-us-west"
EOF

Delete the unneeded backup¶

Deleting a previously saved backup requires not more than the backup name. This name can be taken from the list of available backups returned by the following command:

kubectl get pxc-backup

When the name is known, backup can be deleted as follows:

kubectl delete pxc-backup/<backup-name>

Copy backup to a local machine¶

Make a local copy of a previously saved backup requires not more than the backup name. This name can be taken from the list of available backups returned by the following command:

kubectl get pxc-backup

When the name is known, backup can be downloaded to the local machine as follows:

./deploy/backup/copy-backup.sh <backup-name> path/to/dir

For example, this downloaded backup can be restored to the local installation of Percona Server:

service mysqld stop
rm -rf /var/lib/mysql/*
cat xtrabackup.stream | xbstream -x -C /var/lib/mysql
xtrabackup --prepare --target-dir=/var/lib/mysql
chown -R mysql:mysql /var/lib/mysql
service mysqld start