As not all information changes between each backup, the incremental backup strategy uses this to reduce the storage needs and the duration of making a backup.
An incremental backup copies all pages since a specific LSN.
Once this pages have been put together in their respective order, applying the logs will recreate the process that affected the database, yielding the data at the moment of the most recently created backup.
First, a full backup is needed, this will be the BASE for the incremental one:
$ innobackupex /data/backups
This will create a timestamped directory in /data/backups. Assuming that the backup is done last day of the year, BASEDIR would be /data/backups/2011-12-31_23-01-18, for example.
You can use the innobackupex --no-timestamp option to override this behavior and the backup will be created in the given directory.
If you check at the xtrabackup-checkpoints file in BASE-DIR, you should see something like:
backup_type = full-backuped from_lsn = 0 to_lsn = 1291135
To create an incremental backup the next day, use the --incremental option and provide the BASEDIR:
$ innobackupex --incremental /data/backups --incremental-basedir=BASEDIR
and another timestamped directory will be created in /data/backups, in this example, /data/backups/2012-01-01_23-01-18 containing the incremental backup. We will call this INCREMENTAL-DIR-1.
If you check at the xtrabackup-checkpoints file in INCREMENTAL-DIR-1, you should see something like:
backup_type = incremental from_lsn = 1291135 to_lsn = 1352113
Creating another incremental backup the next day will be analogous, but this time the previous incremental one will be base:
$ innobackupex --incremental /data/backups --incremental-basedir=INCREMENTAL-DIR-1
yielding (in this example) /data/backups/2012-01-02_23-02-08. We will use INCREMENTAL-DIR-2 instead for simplicity.
At this point, the xtrabackup-checkpoints file in INCREMENTAL-DIR-2 should contain something like:
backup_type = incremental from_lsn = 1352113 to_lsn = 1358967
innobackupex --incremental /data/backups --incremental-lsn=1291135 innobackupex --incremental /data/backups --incremental-lsn=1358967
This is a very useful way of doing an incremental backup, since not always the base or the last incremental will be available in the system.
Preparing incremental backups is a bit different than full ones. This is, perhaps, the stage where more attention is needed:
- First, only the committed transactions must be replayed on each backup. This will put the base full backup and the incremental ones altogether.
- Then, the uncommitted transaction must be rolled back in order to have a ready-to-use backup.
If you replay the commit ed transactions and rollback the uncommitted ones on the base backup, you will not be able to add the incremental ones. If you do this on an incremental one, you won’t be able to add data from that moment and the remaining increments.
Having this in mind, the procedure is very straight-forward using the --redo-only option, starting with the base backup:
innobackupex --apply-log --redo-only BASE-DIR
You should see an output similar to:
120103 22:00:12 InnoDB: Shutdown completed; log sequence number 1291135 120103 22:00:12 innobackupex: completed OK!
Then, the first incremental backup can be applied to the base backup, by issuing:
innobackupex --apply-log --redo-only BASE-DIR --incremental-dir=INCREMENTAL-DIR-1
You should see an output similar to the previous one but with corresponding LSN:
120103 22:08:43 InnoDB: Shutdown completed; log sequence number 1358967 120103 22:08:43 innobackupex: completed OK!
If no --incremental-dir is set, innobackupex will use the most recently subdirectory created in the basedir.
At this moment, BASE-DIR contains the data up to the moment of the first incremental backup. Note that the full data will be always in the directory of the base backup, as we are appending the increments to it.
Repeat the procedure with the second one:
innobackupex --apply-log BASE-DIR --incremental-dir=INCREMENTAL-DIR-2
If the “completed OK!” message was shown, the final data will be in the base backup directory, BASE-DIR.
You can use this procedure to add more increments to the base, as long as you do it in the chronological order that the backups were done. If you omit this order, the backup will be useless. If you have doubts about the order that they must be applied, you can check the file xtrabackup_checkpoints at the directory of each one, as shown in the beginning of this section.
Once you put all the parts together, you can prepare again the full backup (base + incrementals) once again to rollback the uncommitted transactions:
innobackupex --apply-log BASE-DIR
Now your backup is ready to be used immediately after restoring it. This preparation step is “optional”, as if you restore it without doing it, the database server will assume that a crash occurred and will begin to rollback the uncommitted transaction (causing some downtime which can be avoided).
Note that the iblog* files will not be created by innobackupex, if you want them to be created, use xtrabackup –prepare on the directory. Otherwise, the files will be created by the server once started.
After preparing the incremental backups, the base directory contains the same as a full one. For restoring it you can use:
innobackupex --copy-back BASE-DIR
and you may have to change the ownership as detailed on Restoring a Full Backup with innobackupex.
For general inquiries, please send us your question and someone will contact you.