This talk is about measuring and reducing noise in benchmark results. Properly tuning the operating system and hardware to achieve stable results in benchmarks becomes an art in itself these days. There may be many reasons for that:
- jitter in CPU and I/O schedulers
- dynamic CPU frequency scaling
- process address space randomization
- kernel configuration
If you are not seeing stable results in your performance comparisons, you are wasting your time. Since I do a lot of MySQL benchmarks as a part of my job, I have collected a number of recipes to measure and reduce system noise and achieve more stable numbers in benchmarks. I'm going to describe those recipes as well as the new sysbench module implemented to automate those tasks and simplify system tuning for other people.
Alexey worked at MySQL AB/Sun/Oracle as a Performance Engineer in the High Performance group and Software Engineer in the MySQL development team. He also worked at Percona on Percona Server, XtraBackup and XtraDB cluster. He is also the maintainer of sysbench, a popular benchmark tool.