Benchmark Noise Reduction: How to Configure Your Machines for Stable Results

This talk is about measuring and reducing noise in benchmark results. Properly tuning the operating system and hardware to achieve stable results in benchmarks becomes an art in itself these days. There may be many reasons for that:

- jitter in CPU and I/O schedulers
- dynamic CPU frequency scaling
- process address space randomization
- kernel configuration
- etc.

If you are not seeing stable results in your performance comparisons, you are wasting your time. Since I do a lot of MySQL benchmarks as a part of my job, I have collected a number of recipes to measure and reduce system noise and achieve more stable numbers in benchmarks. I'm going to describe those recipes as well as the new sysbench module implemented to automate those tasks and simplify system tuning for other people.