Backing up binary log files with mysqlbinlog

January 18, 2012

Author

Tamas Kozak

Insight for DBAs

MySQL

Share this Post:

Backing up binary logs are essential part of creating good backup infrastructure as it gives you the possibility for point in time recovery. After restoring a database from backup you have the option to recover changes that happend after taking a backup. The problem with this approach was that you had to do periodic filesystem level backups of the binary log files which could still lead to data loss depending on the interval you back them up.
Recently in MySQL 5.6, mysqlbinlog got a new feature addition that supports connecting to remote MySQL instances and dumping binary log data to local disks ( http://dev.mysql.com/doc/refman/5.6/en/mysqlbinlog-backup.html ). This can be used as a foundation of our live binary log backups.

The wrapper script below will connect to the remote server specified in the config and ensure mysqlbinlog utility is up and running. By default if you do not supply the binary log file, mysqlbinlog deletes and overwrites them all that is undesired behaviour in our case, so we have to supply the name of the last binary log. This last file will be still overwritten hence we make a backup first.

#!/bin/sh<br><br>source $1<br>cd $BACKUPDIR<br><br>echo "Backup dir: $BACKUPDIR "<br><br>while :<br>do<br>LASTFILE=`ls -1 $BACKUPDIR|grep -v orig|tail -n 1`<br>TIMESTAMP=`date +%s`<br>FILESIZE=$(stat -c%s "$LASTFILE")<br><br>if [ $FILESIZE -gt 0 ]; then<br>    echo "Backing up last binlog"<br>    mv $LASTFILE $LASTFILE.orig$TIMESTAMP<br>fi<br>touch $LASTFILE<br>echo "Starting live binlog backup"<br>$MBL --raw --read-from-remote-server --stop-never --host $MYSQLHOST --port $MYSQLPORT -u $MYSQLUSER -p$MYSQLPASS $LASTFILE <br><br>echo "mysqlbinlog exited with $? trying to reconnect in $RESPAWN seconds."<br><br>sleep $RESPAWN<br>done

1

#!/bin/sh source $1 cd $BACKUPDIR echo "Backup dir: $BACKUPDIR " while : do LASTFILE=`ls -1 $BACKUPDIR|grep -v orig|tail -n 1` TIMESTAMP=`date +%s` FILESIZE=$(stat -c%s "$LASTFILE") if [ $FILESIZE -gt 0 ]; then echo "Backing up last binlog" mv $LASTFILE $LASTFILE.orig$TIMESTAMP fi touch $LASTFILE echo "Starting live binlog backup" $MBL --raw --read-from-remote-server --stop-never --host $MYSQLHOST --port $MYSQLPORT -u $MYSQLUSER -p$MYSQLPASS $LASTFILE echo "mysqlbinlog exited with $? trying to reconnect in $RESPAWN seconds." sleep $RESPAWN done

Configuration file:

MBL=/opt/mysql5.6/usr/bin/mysqlbinlog <br>MYSQLHOST=10.10.10.10<br>MYSQLPORT=3306<br>MYSQLUSER=replication_user<br>MYSQLPASS=replication_pass<br>BACKUPDIR=/media/binlogs/server2/<br><br># time to wait before reconnecting after failure<br>RESPAWN=10

1	MBL=/opt/mysql5.6/usr/bin/mysqlbinlog <br>MYSQLHOST=10.10.10.10<br>MYSQLPORT=3306<br>MYSQLUSER=replication_user<br>MYSQLPASS=replication_pass<br>BACKUPDIR=/media/binlogs/server2/<br><br># time to wait before reconnecting after failure<br>RESPAWN=10

Starting in the background with logging to /var/log/livebinlog/server2.log:

nohup /media/binlogs/livebinlog.sh /media/binlogs/livebackup.server2.conf 2&gt;&amp;1 &gt; /var/log/livebinlog/server2.log &amp;

1	nohup /media/binlogs/livebinlog.sh /media/binlogs/livebackup.server2.conf 2>&1 > /var/log/livebinlog/server2.log &

As a great addition, older logfiles that have been rotated can be checked against the MySQL server’s version if they are the same or not. For this purpose you can use rsync in “dry-run” mode.

Please note MySQL 5.6 is not yet released as GA but you can use mysqlbinlog to backup MySQL 5.1 and 5.5 databases.