Over a year ago I wrote about pretty nasty Innodb Recovery Bug. I ran in the same situation again (different system, different customer) and went to see the status of the bug… and it is still open.
You may thing it is minor issue but in fact with large buffer pool this bug makes database virtually unrecoverable (if 10% of progress in 2hours qualifies as that). It is especially nasty as it is quite hard to predict. Both customers had MySQL crash recovery happening in reasonable time… most of the times until they run into this problem.
So what is the point ? Have modest expectations about when your favorite MySQL bugs are fixed (This is actually Innodb one, so Innobase/Oracle is responsible for fixing it not MySQL/Sun but there are MySQL bugs not fixed for years too). Look for workarounds or ways to fix things yourself.
In particular case workaround was rather easy – reducing Innodb buffer pool size to 4GB instead of 24G and disabling innodb_flush_method=O_DIRECT so OS cache can be used for IOs. This made database to complete crash recovery in 30 minutes.
