Announcement

Announcement Module
Collapse
No announcement yet.

Backup fails with "log block numbers mismatch"

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Backup fails with "log block numbers mismatch"

    Hi,
    I realize that the "log block numbers mismatch" error is a relativly common error and that it has been reported many times before but I cannot seem to get around the problem with the suggested fixes. The backup sometimes succeeds, probably about 30% of the times it is run, which is obviously not good enough.

    I have tried to increase the innodb_log_file_size so that the log should contain a few hours of data and I have made sure that write IOPS isn't a problem by trying to backup to a ramdisk. The server should be plenty powerful with Intel DC3700 SSD RAID1 for data and 15k SAS RAID10 with BBU for logs (also 12 cores latest gen Xeon and 128GB RAM).

    The dataset is small at about 32GB but is updated frequently and the disks writes about a TB per day in total.

    From what I understand the log size that I have now of 2GB should be plenty, perhaps even excessive, and I see no reason log copying should be too slow with these disks.

    Any ideas what I can try?

    Output of xtrabackup below.

    Regards,
    Patrik


    === Backing up database ===
    Thu Apr 17 02:00:01 CEST 2014

    InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
    and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.

    This software is published under
    the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.

    Get the latest version of Percona XtraBackup, documentation, and help resources:
    http://www.percona.com/xb/p

    140417 02:00:02 innobackupex: Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_file=/root/.my.cnf;mysql_read_default_group=xtrabackup' (using password: NO).
    140417 02:00:02 innobackupex: Connected to MySQL server
    140417 02:00:02 innobackupex: Executing a version check against the server...
    140417 02:00:06 innobackupex: Done.
    IMPORTANT: Please check that the backup run completes successfully.
    At the end of a successful backup run innobackupex
    prints "completed OK!".

    innobackupex: Using mysql server version 5.6.15-63.0-log

    innobackupex: Created backup directory /root

    140417 02:00:06 innobackupex: Starting ibbackup with command: xtrabackup_56 --defaults-extra-file="/root/.my.cnf" --defaults-group="mysqld" --backup --suspend-at-end --target-dir=/var/tmp/mem --tmpdir=/var/tmp/mem --tables_file='/var/tmp/mem/backup/tables_to_backup.txt' --stream=tar
    innobackupex: Waiting for ibbackup (pid=26846) to suspend
    innobackupex: Suspend file '/var/tmp/mem/xtrabackup_suspended_2'

    xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: 733)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to /var/lib/mysql/
    xtrabackup: using the following InnoDB configuration:
    xtrabackup: innodb_data_home_dir =
    xtrabackup: innodb_data_file_path = /opt/mysql/ibdata1:10M:autoextend:max:10G
    xtrabackup: innodb_log_group_home_dir = /opt/mysql/
    xtrabackup: innodb_log_files_in_group = 2
    xtrabackup: innodb_log_file_size = 2147483648
    xtrabackup: using O_DIRECT
    xtrabackup: error: log block numbers mismatch:
    xtrabackup: error: expected log block no. 507603581, but got no. 515992181 from the log file.
    xtrabackup: error: it looks like InnoDB log has wrapped around before xtrabackup could process all records due to either log copying being too slow, or log files being too small.
    xtrabackup: Error: xtrabackup_copy_logfile() failed.
    innobackupex: Error: The xtrabackup child process has died at /usr/bin/innobackupex line 2622.

    === Backing up /etc ===
    Thu Apr 17 02:05:38 CEST 2014


  • #2
    We had some related bug reports in the past:

    https://bugs.launchpad.net/percona-x...p/+bug/1050355
    https://bugs.launchpad.net/percona-x...p/+bug/1206309

    I would suggest to change backup time probably during off peak time. Also i would suggest to configure good redo log file size as per http://www.mysqlperformanceblog.com/...log-file-size/

    Comment

    Working...
    X