]]>
]]>

How to debug lock (hang)

If MySQL stopped to response you may get information about where it hang by next steps:

  • attach gdb to process
 gdb -p `pidof mysqld`   

you will see output like :

                                                                                             
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Attaching to process 21172
Reading symbols from /usr/local/mysql-5.1.35-xtradb6-new/libexec/mysqld...done.
Reading symbols from /lib/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7ff46521c6f0 (LWP 21172)]
[New Thread 0x54037950 (LWP 22724)]
[New Thread 0x41d1a950 (LWP 22723)]
[New Thread 0x41c16950 (LWP 22721)]
[New Thread 0x41c98950 (LWP 22719)]
[New Thread 0x53fb5950 (LWP 22717)]
[New Thread 0x41cd9950 (LWP 22716)]
[New Thread 0x53e2f950 (LWP 22715)]
[New Thread 0x41c57950 (LWP 22711)]
[New Thread 0x53f74950 (LWP 22710)]
[New Thread 0x53f33950 (LWP 22709)]
[New Thread 0x53ef2950 (LWP 22707)]
[New Thread 0x53ff6950 (LWP 22704)]
[New Thread 0x53dee950 (LWP 22702)]
[New Thread 0x53e70950 (LWP 22700)]
[New Thread 0x53dad950 (LWP 22699)]
[New Thread 0x53eb1950 (LWP 22698)]
[New Thread 0x41bd5950 (LWP 21215)]
[New Thread 0x53d6c950 (LWP 21214)]
[New Thread 0x5356b950 (LWP 21213)]
[New Thread 0x52d6a950 (LWP 21212)]
[New Thread 0x52569950 (LWP 21207)]
[New Thread 0x51d68950 (LWP 21206)]
[New Thread 0x51567950 (LWP 21205)]
[New Thread 0x50d66950 (LWP 21204)]
[New Thread 0x50565950 (LWP 21203)]
[New Thread 0x4fd64950 (LWP 21202)]
[New Thread 0x4f563950 (LWP 21201)]
[New Thread 0x4ed62950 (LWP 21200)]
[New Thread 0x4e561950 (LWP 21199)]
[New Thread 0x4dd60950 (LWP 21198)]
[New Thread 0x4d55f950 (LWP 21197)]
[New Thread 0x4cd5e950 (LWP 21196)]
[New Thread 0x4c55d950 (LWP 21195)]
[New Thread 0x4bd5c950 (LWP 21194)]
[New Thread 0x4b55b950 (LWP 21193)]
[New Thread 0x4ad5a950 (LWP 21192)]
[New Thread 0x4a559950 (LWP 21191)]
[New Thread 0x49d58950 (LWP 21190)]
[New Thread 0x49557950 (LWP 21189)]
[New Thread 0x48d56950 (LWP 21188)]
[New Thread 0x48555950 (LWP 21187)]
[New Thread 0x47d54950 (LWP 21186)]
[New Thread 0x47553950 (LWP 21185)]
[New Thread 0x46d52950 (LWP 21184)]
[New Thread 0x46551950 (LWP 21183)]
[New Thread 0x45d50950 (LWP 21182)]
[New Thread 0x4554f950 (LWP 21181)]
[New Thread 0x44d4e950 (LWP 21180)]
[New Thread 0x4454d950 (LWP 21179)]
[New Thread 0x43d4c950 (LWP 21178)]
[New Thread 0x4354b950 (LWP 21177)]
[New Thread 0x42d4a950 (LWP 21176)]
[New Thread 0x42549950 (LWP 21175)]
[New Thread 0x41393950 (LWP 21174)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/libstdc++.so.6...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
0x00007ff463b47482 in select () from /lib/libc.so.6
  • show current threads
 (gdb) info threads
  55 Thread 0x41393950 (LWP 21174)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  54 Thread 0x42549950 (LWP 21175)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  53 Thread 0x42d4a950 (LWP 21176)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  52 Thread 0x4354b950 (LWP 21177)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  51 Thread 0x43d4c950 (LWP 21178)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  50 Thread 0x4454d950 (LWP 21179)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  49 Thread 0x44d4e950 (LWP 21180)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  48 Thread 0x4554f950 (LWP 21181)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  47 Thread 0x45d50950 (LWP 21182)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  46 Thread 0x46551950 (LWP 21183)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  45 Thread 0x46d52950 (LWP 21184)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  44 Thread 0x47553950 (LWP 21185)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  43 Thread 0x47d54950 (LWP 21186)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  42 Thread 0x48555950 (LWP 21187)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  41 Thread 0x48d56950 (LWP 21188)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  40 Thread 0x49557950 (LWP 21189)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  39 Thread 0x49d58950 (LWP 21190)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  38 Thread 0x4a559950 (LWP 21191)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  37 Thread 0x4ad5a950 (LWP 21192)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  36 Thread 0x4b55b950 (LWP 21193)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  35 Thread 0x4bd5c950 (LWP 21194)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  34 Thread 0x4c55d950 (LWP 21195)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  33 Thread 0x4cd5e950 (LWP 21196)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  32 Thread 0x4d55f950 (LWP 21197)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  31 Thread 0x4dd60950 (LWP 21198)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  30 Thread 0x4e561950 (LWP 21199)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  29 Thread 0x4ed62950 (LWP 21200)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  28 Thread 0x4f563950 (LWP 21201)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  27 Thread 0x4fd64950 (LWP 21202)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  26 Thread 0x50565950 (LWP 21203)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  25 Thread 0x50d66950 (LWP 21204)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  24 Thread 0x51567950 (LWP 21205)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  23 Thread 0x51d68950 (LWP 21206)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  22 Thread 0x52569950 (LWP 21207)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  21 Thread 0x52d6a950 (LWP 21212)  0x00007ff463b47482 in select () from /lib/libc.so.6
  20 Thread 0x5356b950 (LWP 21213)  0x00007ff463b47482 in select () from /lib/libc.so.6
  19 Thread 0x53d6c950 (LWP 21214)  0x00007ff463b47482 in select () from /lib/libc.so.6
  18 Thread 0x41bd5950 (LWP 21215)  0x00007ff464dffe7b in sigwait () from /lib/libpthread.so.0
  17 Thread 0x53eb1950 (LWP 22698)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  16 Thread 0x53dad950 (LWP 22699)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  15 Thread 0x53e70950 (LWP 22700)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  14 Thread 0x53dee950 (LWP 22702)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  13 Thread 0x53ff6950 (LWP 22704)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  12 Thread 0x53ef2950 (LWP 22707)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  11 Thread 0x53f33950 (LWP 22709)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  10 Thread 0x53f74950 (LWP 22710)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  9 Thread 0x41c57950 (LWP 22711)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  8 Thread 0x53e2f950 (LWP 22715)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  7 Thread 0x41cd9950 (LWP 22716)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  6 Thread 0x53fb5950 (LWP 22717)  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  5 Thread 0x41c98950 (LWP 22719)  0x000000000078594f in lock_rec_convert_impl_to_expl ()
  4 Thread 0x41c16950 (LWP 22721)  0x00000000007a0fc0 in page_check_dir ()
  3 Thread 0x41d1a950 (LWP 22723)  0x000000000052adf0 in Item_func_eq::val_int ()
  2 Thread 0x54037950 (LWP 22724)  0x00007ff464dfef4b in read () from /lib/libpthread.so.0
  1 Thread 0x7ff46521c6f0 (LWP 21172)  0x00007ff463b47482 in select () from /lib/libc.so.6
Current language:  auto; currently asm
  • get backtrace for each thread
 (gdb) thread a all bt
Thread 55 (Thread 0x41393950 (LWP 21174)):
#0  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0x000000000079b2bd in os_event_wait_low ()
#2  0x00000000007990e9 in os_aio_simulated_handle ()
#3  0x0000000000763b11 in fil_aio_wait ()
#4  0x00000000007dcf1c in io_handler_thread ()
#5  0x00007ff464df83ea in start_thread () from /lib/libpthread.so.0
#6  0x00007ff463b4ec6d in clone () from /lib/libc.so.6
#7  0x0000000000000000 in ?? ()

Thread 54 (Thread 0x42549950 (LWP 21175)):
#0  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0x000000000079b2bd in os_event_wait_low ()
#2  0x00000000007990e9 in os_aio_simulated_handle ()
#3  0x0000000000763b11 in fil_aio_wait ()
#4  0x00000000007dcf1c in io_handler_thread ()
#5  0x00007ff464df83ea in start_thread () from /lib/libpthread.so.0
#6  0x00007ff463b4ec6d in clone () from /lib/libc.so.6
#7  0x0000000000000000 in ?? ()

Thread 53 (Thread 0x42d4a950 (LWP 21176)):
#0  0x00007ff464dfc2d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0x000000000079b2bd in os_event_wait_low ()
#2  0x00000000007990e9 in os_aio_simulated_handle ()
#3  0x0000000000763b11 in fil_aio_wait ()
#4  0x00000000007dcf1c in io_handler_thread ()
#5  0x00007ff464df83ea in start_thread () from /lib/libpthread.so.0
#6  0x00007ff463b4ec6d in clone () from /lib/libc.so.6
#7  0x0000000000000000 in ?? ()
 
howto/debug_lock.txt · Last modified: 2011/07/17 21:05 (external edit)
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Contact Us 24 Hours A Day
SupportContact us 24×7
Emergency? Contact us for help now!
Sales North America(888) 316-9775 or
(208) 473-2904
Sales
Europe
+44-208-133-0309 (UK)
0-800-051-8984 (UK)
0-800-181-0665 (GER)
Training(855) 55TRAIN or
(925) 271-5054

 

Share This
]]> ]]>