September 20, 2014

How quickly you should expect to see bugs fixed

Over a year ago I wrote about pretty nasty Innodb Recovery Bug. I ran in the same situation again (different system, different customer) and went to see the status of the bug… and it is still open.

You may thing it is minor issue but in fact with large buffer pool this bug makes database virtually unrecoverable (if 10% of progress in 2hours qualifies as that). It is especially nasty as it is quite hard to predict. Both customers had MySQL crash recovery happening in reasonable time… most of the times until they run into this problem.

So what is the point ? Have modest expectations about when your favorite MySQL bugs are fixed (This is actually Innodb one, so Innobase/Oracle is responsible for fixing it not MySQL/Sun but there are MySQL bugs not fixed for years too). Look for workarounds or ways to fix things yourself.

In particular case workaround was rather easy – reducing Innodb buffer pool size to 4GB instead of 24G and disabling innodb_flush_method=O_DIRECT so OS cache can be used for IOs. This made database to complete crash recovery in 30 minutes.

About Peter Zaitsev

Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.

Comments

  1. gigiduru says:

    I feel at ease with the recovery time being reduced to only 30 minutes. Yet, it’s enough time to get fired and even be replaced by the next MySQL DBA in the queue.

  2. peter says:

    Hi,

    I think numbers are not that important here. The process is. For someone 30 minutes may be OK for others 5 minutes may be too much. As professional MySQL DBA you need to learn these requirements from the business and make sure to have architecture and processes to support them.

    Also this will make business people to agree on lower guarantees in many cases because higher guarantees can just cost too much :)

  3. gigiduru says:

    Yeah, sure, the recovery time really varies from company to company as the SLA varies as well. However, my line was more of a sting than a complain.

    It would be extremely nice to see that your company would have changed the logo from “Everything about MySQL Performance” to “Everything about Oracle/Postgresql/Any-other-RDBMS Performance”, in this way your company’s resources/brains pool would be much better put to work.

    Sorry, I couldn’t help it, I had to spill it out.

  4. beberlei says:

    i am no expert in innodb, but i have one question regarding this issue. is this related to innodb log file size also? if i would set my log file that small that there isn’t just that much to recover from?

Speak Your Mind

*