Announcement

Announcement Module
Collapse
No announcement yet.

File name ./ib_logfile0 InnoDB: File operation call: 'open' returned OS error 71.

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • File name ./ib_logfile0 InnoDB: File operation call: 'open' returned OS error 71.

    === Problem solved. See last comment. ===

    Like others, I have encountered an error with innobackupex stating cannot find path specified. I tried both version 2.1.7 and 2.0.8 with similar results. I am running MySQL 5.6.15 on RHEL 6.4. I have not used innobackupex on this machine before, although I have used it against 5.5 dbs on other machines.

    2.1.7 reported
    ------------------
    xtrabackup_56 version 2.1.7 for MySQL server 5.6.15 Linux (x86_64) (revision id: 721)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to
    xtrabackup: using the following InnoDB configuration:
    xtrabackup: innodb_data_home_dir = ./
    xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
    xtrabackup: innodb_log_group_home_dir = ./
    xtrabackup: innodb_log_files_in_group = 2
    xtrabackup: innodb_log_file_size = 50331648
    2014-02-18 15:55:31 7f38abdd7720 InnoDB: Operating system error number 2 in a file operation.
    InnoDB: The error means the system cannot find the path specified.
    InnoDB: File ./ib_logfile0: 'open' returned OS error 71. Cannot continue operation
    innobackupex: Error: The xtrabackup child process has died at /usr/bin/innobackupex line 2648.


    2.0.8 reported
    ------------------
    xtrabackup_56 version 2.0.8 for MySQL server 5.6.10 Linux (x86_64) (revision id: 587)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to
    xtrabackup: Target instance is assumed as followings.
    xtrabackup: innodb_data_home_dir = ./
    xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
    xtrabackup: innodb_log_group_home_dir = ./
    xtrabackup: innodb_log_files_in_group = 2
    xtrabackup: innodb_log_file_size = 50331648
    2014-02-18 15:59:33 7ffe84979720 InnoDB: Operating system error number 2 in a file operation.
    InnoDB: The error means the system cannot find the path specified.
    2014-02-18 15:59:33 7ffe84979720 InnoDB: File name ./ib_logfile0
    2014-02-18 15:59:33 7ffe84979720 InnoDB: File operation call: 'open' returned OS error 71.
    2014-02-18 15:59:33 7ffe84979720 InnoDB: Cannot continue operation.
    innobackupex: Error: ibbackup child process has died at /usr/bin/innobackupex line 386.

    The /etc/my.cnf file
    ------------------------
    [mysqld56]
    ## list temporary variables or settings here ##
    log_warnings = 2
    ##
    skip-slave-start
    basedir = /usr/local/mysql56/
    binlog-checksum = NONE
    binlog_format = MIXED
    connect_timeout = 60
    datadir = /db01/mysql56/
    default-storage-engine = InnoDB
    event_scheduler = ON
    expire_logs_days = 7
    group_concat_max_len = 99999
    innodb_autoextend_increment=64
    innodb_buffer_pool_size = 5G
    innodb_data_home_dir = /db01/mysql56
    innodb_file_format = barracuda
    innodb_file_per_table
    innodb_flush_log_at_trx_commit = 1
    innodb_io_capacity = 200
    innodb_log_file_size = 64M
    innodb_log_files_in_group=3
    innodb_log_group_home_dir=/db01/mysql56/
    innodb_purge_threads = 1
    innodb_stats_on_metadata= 0
    join_buffer_size = 4000000
    key_buffer_size = 256000000
    local-infile = 0
    log-bin = mysql-bin
    log-error = /var/log/mysqld56.log
    log-queries-not-using-indexes
    log-slave-updates
    long-query-time = 5
    master-info-repository = TABLE
    max_allowed_packet = 32M
    max_connect_errors = 9999
    max_connections = 400
    max_heap_table_size = 64000000
    myisam-recover = FORCE,BACKUP
    mysqladmin = /usr/local/mysql56/bin/mysqladmin
    mysqld = /usr/local/mysql56/bin/mysqld
    net_read_timeout = 120
    old_passwords = 0
    open_files_limit = 4410
    performance_schema = 1
    pid-file = /var/run/mysqld/mysqld56.pid
    plugin_dir = /opt/mysql56/plugins
    port = 3306
    query_cache_size = 16777216
    query_cache_type = ON
    relay-log = /db01/mysql56/relay-bin
    relay-log-index = /db01/mysql56/relay-bin.index
    relay-log-info-file = /db01/mysql56/relay-bin.info
    secure_auth = OFF
    server-id = 3002
    slave_net_timeout = 60
    slow_query_log
    slow_query_log_file = /var/log/mysqld56-slow.log
    socket = /var/run/mysqld/mysql.sock
    sort_buffer_size = 192000
    symbolic-links = 0
    sync_binlog = 1
    table_open_cache = 2000
    table_open_cache = 2000
    thread_cache_size = 108
    tmp_table_size = 64000000
    user = mysql

    Hope you can suggest how I can get the backup to run.

    Thank you!
    -dale
    Last edited by daleb; 02-20-2014, 05:03 PM.

  • #2
    Forgot to mention, this is on RHEL6. Everywhere else I am running MySQL 5.5 on RHEL5 and not encountering any problem with Xtrabackup/Innobackupex. Very satisfied with the product, excepting this little hiccup. :-)

    Comment


    • #3
      Can you please mention all your steps of the backup. Also, what's the output of

      $ xtrabackup --print-defaults

      Comment


      • #4
        By the way, the only difference i can see is innodb_log_files_in_group. Xtrabackup took it as innodb_log_files_in_group = 2 while in your my.cnf it set at innodb_log_files_in_group = 3. We need all your steps of backup to drill down the problem. Either it's happen on during taking backup or at apply-log stage.

        Comment


        • #5
          The output displays no arguments.

          # xtrabackup --print-defaults
          xtrabackup would have been started with the following arguments:

          #

          Comment


          • #6
            <username and password obfuscated for obvious security reasons>

            Ran this command:
            innobackupex --user='xxxxx' --socket=/var/run/mysqld/mysql.sock --password='xxxxx' --defaults-group=mysqld56 /db01/perconabackups

            Output of run:

            InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
            and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.

            This software is published under
            the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.

            140220 13:49:46 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='xxxxx' --socket='/var/run/mysqld/mysql.sock' --unbuffered --
            140220 13:49:46 innobackupex: Connected to database with mysql child process (pid=29490)
            140220 13:49:52 innobackupex: Connection to database server closed
            IMPORTANT: Please check that the backup run completes successfully.
            At the end of a successful backup run innobackupex
            prints "completed OK!".

            innobackupex: Using mysql Ver 14.14 Distrib 5.1.69, for redhat-linux-gnu (x86_64) using readline 5.1
            innobackupex: Using mysql server version Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

            innobackupex: Created backup directory /db01/perconabackups/2014-02-20_13-49-52
            140220 13:49:52 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='xxxxx' --socket='/var/run/mysqld/mysql.sock' --unbuffered --
            140220 13:49:52 innobackupex: Connected to database with mysql child process (pid=29513)
            140220 13:49:54 innobackupex: Connection to database server closed

            140220 13:49:54 innobackupex: Starting ibbackup with command: xtrabackup_56 --defaults-group="mysqld56" --backup --suspend-at-end --target-dir=/db01/perconabackups/2014-02-20_13-49-52 --tmpdir=/tmp
            innobackupex: Waiting for ibbackup (pid=29519) to suspend
            innobackupex: Suspend file '/db01/perconabackups/2014-02-20_13-49-52/xtrabackup_suspended'

            xtrabackup_56 version 2.0.8 for MySQL server 5.6.10 Linux (x86_64) (revision id: 587)
            xtrabackup: uses posix_fadvise().
            xtrabackup: cd to
            xtrabackup: Target instance is assumed as followings.
            xtrabackup: innodb_data_home_dir = ./
            xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
            xtrabackup: innodb_log_group_home_dir = ./
            xtrabackup: innodb_log_files_in_group = 2
            xtrabackup: innodb_log_file_size = 50331648
            2014-02-20 13:49:54 7fb4bac75720 InnoDB: Operating system error number 2 in a file operation.
            InnoDB: The error means the system cannot find the path specified.
            2014-02-20 13:49:54 7fb4bac75720 InnoDB: File name ./ib_logfile0
            2014-02-20 13:49:54 7fb4bac75720 InnoDB: File operation call: 'open' returned OS error 71.
            2014-02-20 13:49:54 7fb4bac75720 InnoDB: Cannot continue operation.
            innobackupex: Error: ibbackup child process has died at /usr/bin/innobackupex line 386.

            #

            Comment


            • #7
              Mirfan,

              I also opened Bug #1282270 if you want to note anything there.

              Thank you!
              -dale
              Last edited by daleb; 02-20-2014, 04:59 PM.

              Comment


              • #8
                Found the problem in my install. The /etc/my.cnf owner was root, group root, permissions 640. Changed ownership of file to mysql, left group root and left permissions alone.

                The hint was that you pointed out that the log file count was 2, not 3 as in my.cnf. Looked at /etc/my.cnf (ls -l /etc/my.cnf) and compared with system where innobackupex worked. Realized I had not set ownership properly.

                Sorry for the interruption!
                Last edited by daleb; 02-20-2014, 05:02 PM.

                Comment


                • #9
                  I have the same issues when working with mariadb 10 on Centos 6.5(Master)
                  I use xtrabackup to dump db from it to 2 Ubuntu 14(mariadb 10 - Slave )
                  I have already set :
                  -rwxrwxrwx 1 mysql root 5088 Oct 28 10:01 /etc/my.cnf
                  But no lucky

                  Comment


                  • #10
                    Can you please share the command you run to took backup along with backup log and my.cnf file

                    Comment


                    • #11
                      Yes,
                      On Master(Centos 6.5 - Mariadb 10.0)
                      Code:
                      innobackupex --user=root --password=password --stream=xbstream ./ |ssh root@10.10.10.220 "xbstream -x -C /var/lib/mysql"
                      and the error :
                      Code:
                      InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
                      and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.
                      
                      This software is published under
                      the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
                      
                      141028 10:20:05 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='root' --unbuffered --
                      141028 10:20:05 innobackupex: Connected to database with mysql child process (pid=2712)
                      141028 10:20:11 innobackupex: Connection to database server closed
                      IMPORTANT: Please check that the backup run completes successfully.
                      At the end of a successful backup run innobackupex
                      prints "completed OK!".
                      
                      innobackupex: Using mysql Ver 15.1 Distrib 10.0.14-MariaDB, for Linux (x86_64) using readline 5.1
                      Warning: World-writable config file '/etc/my.cnf' is ignored
                      innobackupex: Using mysql server version Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
                      
                      Warning: World-writable config file '/etc/my.cnf' is ignored
                      innobackupex: Created backup directory /root
                      141028 10:20:11 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='root' --unbuffered --
                      xbstream: Can't create/write to file './backup-my.cnf' (Errcode: 17)
                      xbstream: failed to create file.
                      141028 10:20:11 innobackupex: Connected to database with mysql child process (pid=2739)
                      141028 10:20:13 innobackupex: Connection to database server closed
                      
                      141028 10:20:13 innobackupex: Starting ibbackup with command: xtrabackup_56 --defaults-group="mysqld" --backup --suspend-at-end --target-dir=/tmp --tmpdir=/tmp --stream=xbstream
                      innobackupex: Waiting for ibbackup (pid=2746) to suspend
                      innobackupex: Suspend file '/tmp/xtrabackup_suspended'
                      
                      Warning: World-writable config file '/etc/my.cnf' is ignored
                      xtrabackup_56 version 2.0.8 for MySQL server 5.6.10 Linux (x86_64) (revision id: 587)
                      xtrabackup: uses posix_fadvise().
                      xtrabackup: cd to
                      xtrabackup: Target instance is assumed as followings.
                      xtrabackup: innodb_data_home_dir = ./
                      xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
                      xtrabackup: innodb_log_group_home_dir = ./
                      xtrabackup: innodb_log_files_in_group = 2
                      xtrabackup: innodb_log_file_size = 50331648
                      2014-10-28 10:20:13 7f8ac7bc5720 InnoDB: Operating system error number 2 in a file operation.
                      InnoDB: The error means the system cannot find the path specified.
                      2014-10-28 10:20:13 7f8ac7bc5720 InnoDB: File name ./ib_logfile0
                      2014-10-28 10:20:13 7f8ac7bc5720 InnoDB: File operation call: 'open' returned OS error 71.
                      2014-10-28 10:20:13 7f8ac7bc5720 InnoDB: Cannot continue operation.
                      innobackupex: Error: ibbackup child process has died at /usr/bin/innobackupex line 386.
                      After review i saw the problem is :
                      Code:
                      xbstream: Can't create/write to file './backup-my.cnf' (Errcode: 17)
                      Code:
                      root@cluster02:/home/itech# ll /var/lib/mysql/
                      total 12
                      drwxr-xr-x  2 mysql mysql 4096 Oct 29 03:00 ./
                      drwxr-xr-x 42 root  root  4096 Oct 28 22:08 ../
                      -rw-r-----  1 root  root   238 Oct 29 03:00 backup-my.cnf
                      But after i chown the /var/lib/mysql on the Slave from root:root to mysql:mysql
                      Code:
                      root@cluster02:/home/itech# chown -R mysql:mysql  /var/lib/mysql/
                      But still error event i chmod -R 777 /var/lib/mysql



                      Last edited by lamletoi; 10-29-2014, 04:09 AM.

                      Comment


                      • #12
                        When i switched a Ubuntu node to became the master and the backup was successfull
                        Code:
                        xtrabackup: Stopping log copying thread.
                        ..>> log scanned up to (1616807)
                        
                        xtrabackup: Creating suspend file '/tmp/xtrabackup_log_copied' with pid '2235'
                        xtrabackup: Transaction log of lsn (1616807) to (1616807) was copied.
                        141029 03:23:34  innobackupex: All tables unlocked
                        
                        innobackupex: Backup created in directory '/home/itech'
                        innobackupex: MySQL binlog position: filename 'mariadb-bin.000013', position 328
                        141029 03:23:34  innobackupex: Connection to database server closed
                        141029 03:23:34  innobackupex: completed OK!

                        Comment


                        • #13
                          Here is th my.cnf on Master(Centos)
                          Code:
                          [root@master01 ~]# cat /etc/my.cnf
                          # Example MariaDB config file for medium systems.
                          #
                          # This is for a system with little memory (32M - 64M) where MariaDB plays
                          # an important part, or systems up to 128M where MariaDB is used together with
                          # other programs (such as a web server)
                          #
                          # MariaDB programs look for option files in a set of
                          # locations which depend on the deployment platform.
                          # You can copy this option file to one of those
                          # locations. For information about these locations, do:
                          # 'my_print_defaults --help' and see what is printed under
                          # Default options are read from the following files in the given order:
                          # More information at: http://dev.mysql.com/doc/mysql/en/option-files.html
                          #
                          # In this file, you can use all long options that a program supports.
                          # If you want to know which options a program supports, run the program
                          # with the "--help" option.
                          
                          # The following options will be passed to all MariaDB clients
                          [client]
                          #password       = your_password
                          port            = 3306
                          socket          = /var/lib/mysql/mysql.sock
                          
                          # Here follows entries for some specific programs
                          
                          # The MariaDB server
                          [mysqld]
                          port            = 3306
                          socket          = /var/lib/mysql/mysql.sock
                          skip-external-locking
                          key_buffer_size = 16M
                          max_allowed_packet = 1M
                          table_open_cache = 64
                          sort_buffer_size = 512K
                          net_buffer_length = 8K
                          read_buffer_size = 256K
                          read_rnd_buffer_size = 512K
                          myisam_sort_buffer_size = 8M
                          
                          # Point the following paths to different dedicated disks
                          #tmpdir         = /tmp/
                          
                          # Don't listen on a TCP/IP port at all. This can be a security enhancement,
                          # if all processes that need to connect to mysqld run on the same host.
                          # All interaction with mysqld must be made via Unix sockets or named pipes.
                          # Note that using this option without enabling named pipes on Windows
                          # (via the "enable-named-pipe" option) will render mysqld useless!
                          # 
                          #skip-networking
                          
                          # Replication Master Server (default)
                          # binary logging is required for replication
                          log-bin=mysql-bin
                          
                          # binary logging format - mixed recommended
                          binlog_format=mixed
                          
                          # required unique id between 1 and 2^32 - 1
                          # defaults to 1 if master-host is not set
                          # but will not function as a master if omitted
                          server-id       = 1
                          bind-address    = 0.0.0.0
                          log_bin         = /var/log/mysql/mariadb-bin
                          sync_binlog     = 0
                          expire_logs_days =      10
                          slave_compressed_protocol = 1
                          #transaction-isolation = READ COMMITED
                          
                          # Replication Slave (comment out master section to use this)
                          #
                          # To configure this host as a replication slave, you can choose between
                          # two methods :
                          #
                          # 1) Use the CHANGE MASTER TO command (fully described in our manual) -
                          #    the syntax is:
                          #
                          #    CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
                          #    MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
                          #
                          #    where you replace <host>, <user>, <password> by quoted strings and
                          #    <port> by the master's port number (3306 by default).
                          #
                          #    Example:
                          #
                          #    CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
                          #    MASTER_USER='joe', MASTER_PASSWORD='secret';
                          #
                          # OR
                          #
                          # 2) Set the variables below. However, in case you choose this method, then
                          #    start replication for the first time (even unsuccessfully, for example
                          #    if you mistyped the password in master-password and the slave fails to
                          #    connect), the slave will create a master.info file, and any later
                          #    change in this file to the variables' values below will be ignored and
                          #    overridden by the content of the master.info file, unless you shutdown
                          #    the slave server, delete master.info and restart the slaver server.
                          #    For that reason, you may want to leave the lines below untouched
                          #    (commented) and instead use CHANGE MASTER TO (see above)
                          #
                          # required unique id between 2 and 2^32 - 1
                          # (and different from the master)
                          # defaults to 2 if master-host is set
                          # but will not function as a slave if omitted
                          #server-id       = 2
                          #
                          # The replication master for this slave - required
                          #master-host     =   <hostname>
                          #
                          # The username the slave will use for authentication when connecting
                          # to the master - required
                          #master-user     =   <username>
                          #
                          # The password the slave will authenticate with when connecting to
                          # the master - required
                          #master-password =   <password>
                          #
                          # The port the master is listening on.
                          # optional - defaults to 3306
                          #master-port     =  <port>
                          #
                          # binary logging - not required for slaves, but recommended
                          #log-bin=mysql-bin
                          
                          # Uncomment the following if you are using InnoDB tables
                          #innodb_data_home_dir = /var/lib/mysql
                          #innodb_data_file_path = ibdata1:10M:autoextend
                          #innodb_log_group_home_dir = /var/lib/mysql
                          # You can set .._buffer_pool_size up to 50 - 80 %
                          # of RAM but beware of setting memory usage too high
                          #innodb_buffer_pool_size = 16M
                          #innodb_additional_mem_pool_size = 2M
                          # Set .._log_file_size to 25 % of buffer pool size
                          innodb_log_file_size = 5M
                          #innodb_log_buffer_size = 8M
                          #innodb_flush_log_at_trx_commit = 1
                          #innodb_lock_wait_timeout = 50
                          
                          [mysqldump]
                          quick
                          max_allowed_packet = 16M
                          
                          [mysql]
                          no-auto-rehash
                          # Remove the next comment character if you are not familiar with SQL
                          #safe-updates
                          
                          [myisamchk]
                          key_buffer_size = 20M
                          sort_buffer_size = 20M
                          read_buffer = 2M
                          write_buffer = 2M
                          
                          [mysqlhotcopy]
                          interactive-timeout

                          Comment


                          • #14
                            I think the problem is that the directory structures are not the same

                            Comment

                            Working...
                            X