Announcement

Announcement Module
Collapse
No announcement yet.

xtrabackup failure: Inappropriate ioctl for device

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • xtrabackup failure: Inappropriate ioctl for device

    Hi there,

    I'm running Percona mysql Server version: 5.5.23-55-log Percona Server (GPL), Release 25.3 and xtrabackup version 2.0.0 for Percona Server 5.1.59 unknown-linux-gnu (x86_64) (revision id: undefined).

    I have 5 database servers with this setup, and 4 are completing their backups fine, and one is failing with the error:

    Failed to stream xtrabackup_binary: Inappropriate ioctl for device at /usr/bin/innobackupex-1.5.1 line 301.

    Throughout the log of the backup, there's also errors with exit code 141 like this:
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/password_resets.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.


    I am running all 5 backups with the command: /usr/bin/innobackupex-1.5.1 --defaults-file=/etc/mysql/my.cnf --user=my_user --password=my_pass --throttle=40 --stream=xbstream ./ | pv -q -L40m | su backups -c 'ssh backups@10.54.168.16 "xbstream -x -C /backups/mysql-9/0-days-ago"'

    Does anyone know what might cause this error or how I should go about troubleshooting it?


    Thank you,

    Tim.

    A truncated copy of the end of the backup log:


    >> log scanned up to (1708839913493)
    >> log scanned up to (1708840156643)
    >> log scanned up to (1708840595827)
    >> log scanned up to (1708841052489)
    >> log scanned up to (1708841365198)
    >> log scanned up to (1708841828552)
    >> log scanned up to (1708842195574)
    [01] ...done

    121105 18:37:24 innobackupex-1.5.1: Continuing after ibbackup has suspended
    121105 18:37:24 innobackupex-1.5.1: Starting mysql with options: --defaults-file='/etc/mysql/my.cnf' --password=xxxxxxxx --user='my_user' --unbuffered --
    121105 18:37:24 innobackupex-1.5.1: Connected to database with mysql child process (pid=13585)
    121105 18:37:26 innobackupex-1.5.1: Starting to lock all tables...
    >> log scanned up to (1708842531794)
    >> log scanned up to (1708842747386)
    121105 18:37:36 innobackupex-1.5.1: All tables locked and flushed to disk

    121105 18:37:36 innobackupex-1.5.1: Starting to backup .frm, .MRG, .MYD, .MYI,
    innobackupex-1.5.1: .TRG, .TRN, .ARM, .ARZ, .CSM, .CSV and .opt files in
    innobackupex-1.5.1: subdirectories of '/var/lib/mysql'
    innobackupex-1.5.1: Backing up files '/var/lib/mysql/beta_billing/*.{frm,MYD,MYI,MRG,TRG,TRN,ARM, ARZ,CSM,CSV,opt,par}' (10 files)
    innobackupex-1.5.1: Backing up files '/var/lib/mysql/my_db_staging/*.{frm,MYD,MYI,MRG,TRG,TRN,ARM ,ARZ,CSM,CSV,opt,par}' (54 files)
    xbstream: Can't create/write to file './my_db_staging/user_opt_outs.ibd' (Errcode: 17)
    xbstream: failed to create file.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/user_opt_outs.ibd'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/subscriptions.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/subscription_plans.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/billing_transactions.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/password_resets.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'my_db_staging/tolk_phrases.frm'.
    innobackupex-1.5.1: Backing up files '/var/lib/mysql/mysql/*.{frm,MYD,MYI,MRG,TRG,TRN,ARM,ARZ,CSM ,CSV,opt,par}' (68 files)
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'mysql/db.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'mysql/procs_priv.MYD'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'mysql/plugin.MYD'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.



    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'mysql/tables_priv.MYI'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'mysql/proxies_priv.MYD'.
    innobackupex-1.5.1: Backing up file '/var/lib/mysql/event_tracker/events.frm'
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'event_tracker/events.frm'.
    innobackupex-1.5.1: Backing up file '/var/lib/mysql/event_tracker/db.opt'
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'event_tracker/db.opt'.
    innobackupex-1.5.1: Backing up files '/var/lib/mysql/performance_schema/*.{frm,MYD,MYI,MRG,TRG,TR N,ARM,ARZ,CSM,CSV,opt,par}' (18 files)
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'performance_schema/cond_instances.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.



    innobackupex-1.5.1: Ignoring nonexistent file 'performance_schema/events_waits_summary_global_by_event_nam e.frm'.
    innobackupex-1.5.1: xbstream -c returned with exit code 141.
    innobackupex-1.5.1: Ignoring nonexistent file 'performance_schema/rwlock_instances.frm'.
    121105 18:37:37 innobackupex-1.5.1: Finished backing up .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ, .CSV, .CSM and .opt files

    innobackupex-1.5.1: Resuming ibbackup

    xtrabackup: The latest check point (for incremental): '1708472789748'
    xtrabackup: Stopping log copying thread.
    .>> log scanned up to (1708842747978)

    121105 18:37:39 innobackupex-1.5.1: All tables unlocked
    121105 18:37:39 innobackupex-1.5.1: Connection to database server closed

    innobackupex-1.5.1: Backup created in directory '/usr/local/sbin'
    innobackupex-1.5.1: MySQL binlog position: filename 'mysql-bin.005115', position 76986073
    Failed to stream xtrabackup_binary: Inappropriate ioctl for device at /usr/bin/innobackupex-1.5.1 line 301.

  • #2
    I just confirmed it is actually xtrabackup_55 that's being run - not an xtrabackup that targets 5.1. This is in the log at the start of the backup:


    121105 17:36:10 innobackupex-1.5.1: Starting ibbackup with command: xtrabackup_55 --defaults-file="/etc/mysql/my.cnf" --backup --suspend-at-end --target-dir=/tmp --throttle=40 --stream=xbstream
    innobackupex-1.5.1: Waiting for ibbackup (pid=26889) to suspend
    innobackupex-1.5.1: Suspend file '/tmp/xtrabackup_suspended'

    xtrabackup_55 version 2.0.0 for Percona Server 5.5.16 Linux (x86_64) (revision id: undefined)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to /var/lib/mysql
    xtrabackup: Target instance is assumed as followings.
    xtrabackup: innodb_data_home_dir = ./
    xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
    xtrabackup: innodb_log_group_home_dir = ./
    xtrabackup: innodb_log_files_in_group = 2
    xtrabackup: innodb_log_file_size = 524288000
    121105 17:36:10 InnoDB: Using Linux native AIO
    xtrabackup: using O_DIRECT
    121105 17:36:10 InnoDB: Warning: allocated tablespace 92, old maximum was 9
    >> log scanned up to (1708590976624)
    [01] Streaming ./ibdata1
    [01] ...done

    Comment


    • #3
      This problem was solved by upgrading from xtrabackup 2.0.0 to 2.0.3.

      About a month ago I added a new table which has opt as part of the table name. Xtrabackup 2.0.0 has a bug where files with opt in the name are backed up twice, causing a problem and several other files to be missed, resulting in the log you see above. This Xtrabackup bug was fixed in 2.0.1.

      Comment

      Working...
      X