Emergency

Cluster down > BF-BF X lock conflict

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Cluster down > BF-BF X lock conflict

    Hello,

    today we had a strange problem with our PXC (5.5.41-25.12-855.trusty
    ). All 3 nodes were offline.

    I found something in the error logs:

    BF-BF X lock conflict

    RECORD LOCKS space id 1965277 page no 68 n bits 120 index `PRIMARY` of table `data_mdt`.`V30_55_044` trx id A72E0B521 lock_mode X locks rec but not gap

    07:13:22 UTC - mysqld got signal 6 ;

    This could be because you hit a bug. It is also possible that this binary

    or one of the libraries it was linked against is corrupt, improperly built,

    or misconfigured. This error can also be caused by malfunctioning hardware.

    We will try our best to scrape up some info that will hopefully help

    diagnose the problem, but since we have already crashed,

    something is definitely wrong and this may fail.

    Please help us make Percona XtraDB Cluster better by reporting any

    bugs at https://bugs.launchpad.net/percona-xtradb-cluster




    key_buffer_size=8388608

    read_buffer_size=131072

    max_used_connections=123

    max_threads=153

    thread_count=25

    connection_count=8

    It is possible that mysqld could use up to

    key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 343066 K bytes of memory

    Hope that's ok; if not, decrease some variables in the equation.




    Thread pointer: 0x7f2f9c000990

    Attempting backtrace. You can use the following information to find out

    where mysqld died. If you see no messages after this, something went

    terribly wrong...

    stack_bottom = 7f3d08332a58 thread_stack 0x40000

    /usr/sbin/mysqld(my_print_stacktrace+0x20)[0x7a4db0]

    /usr/sbin/mysqld(handle_fatal_signal+0x36f)[0x693b0f]

    /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f3d3a67c340]

    /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x7f3d39698cc9]

    /lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f3d3969c0d8]

    /usr/sbin/mysqld[0x50d0c4]

    /usr/sbin/mysqld[0x8ed07a]

    /usr/sbin/mysqld[0x8f223d]

    /usr/sbin/mysqld[0x8f2be3]

    /usr/sbin/mysqld[0x8304ea]

    /usr/sbin/mysqld[0x834454]

    /usr/sbin/mysqld[0x80744d]

    /usr/sbin/mysqld[0x80ca53]

    /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEPK14Relay_log_ info+0x64)[0x72cfa4]

    /usr/sbin/mysqld(_ZN21Update_rows_log_event11do_exec_rowEPK1 4Relay_log_info+0x89)[0x72d769]

    /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Rel ay_log_info+0x1e5)[0x723b55]

    /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta +0x6ab)[0x653bab]

    /usr/lib/libgalera_smm.so(+0x196a96)[0x7f3d1ca68a96]

    /usr/lib/libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_t rxEPvPNS_9TrxHandleE+0xce)[0x7f3d1ca6aabe]

    /usr/lib/libgalera_smm.so(_ZN6galera13ReplicatorSMM11proces s_trxEPvPNS_9TrxHandleE+0x10e)[0x7f3d1ca6d72e]

    /usr/lib/libgalera_smm.so(_ZN6galera15GcsActionSource8dispa tchEPvRK10gcs_actionRb+0x1b9)[0x7f3d1ca4d979]

    /usr/lib/libgalera_smm.so(_ZN6galera15GcsActionSource7proce ssEPvRb+0x4c)[0x7f3d1ca4e86c]

    /usr/lib/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_ recvEPv+0x6b)[0x7f3d1ca6d94b]

    /usr/lib/libgalera_smm.so(galera_recv+0x18)[0x7f3d1ca7e9c8]

    /usr/sbin/mysqld[0x6544a8]

    /usr/sbin/mysqld(start_wsrep_THD+0x34b)[0x5261bb]

    /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7f3d3a674182]

    /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f3d3975c47d]




    Trying to get some variables.

    Some pointers may be invalid and cause the dump to abort.

    Query (0): is an invalid pointer

    Connection ID (thread ID): 14
    Status: NOT_KILLED
    You may download the Percona XtraDB Cluster operations manual by visiting

    http://www.percona.com/software/percona-xtradb-cluster/. You may find information

    in the manual which will help you identify the cause of the crash.

    170911 09:13:22 mysqld_safe Number of processes running now: 0

    170911 09:13:22 mysqld_safe WSREP: not restarting wsrep node automatically

    170911 09:13:22 mysqld_safe mysqld from pid file /var/lib/mysql/nsearch11.pid ended
    Two questions:

    - Why stopped our PXC ?
    - I think the error was at 07:13:22 am, put why stopped the nodes at 09:13:22 ?

    Any ideas ? :-/

    best regards
    Patrick
Working...
X