InnoDB benchmarks

There was several changes in InnoDB to fix scalabilty problems, so I ran benchmark to check
new results and also compare overall performance of InnoDB in 5.0 and 5.1 before and after fixes.

Problems in InnoDB that were fixed:

• • Thread trashing issues with count of theads 100+. In this case
    performance of InnoDB degraded dramatically. The problem was in the mutex impelementation
    and was fixed in 5.1.12 (more info about InnoDB mutexes)

• • Scalabilty issue, the well know bug 15815, that was fixed in 5.0.30 and 5.1.14.

So I took for tests:

• • 5.0.27, the release with both problems

• • 5.0.32-bk (snapshot from 26 Dec 2006) – with 15815 bug fix

• • 5.1.12 with fix of thread trashing

• • 5.1.14 with all fixes

• • 5.0.30 just for comparison.

• • 4.1.22 old release, just for comparison of overall performance

All releases were compiled from sources with options:

and next startup options were used:

Description of benchmark

Specially for our needs I developed benchmark suite which support wide range of tables and queries.
The schema and used queries is avaiable here:
http://docs.google.com/View?docid=d7fzh8b_7dwd8m7
or on our page
https://www.percona.com/blog/files/benchmarks/phptestsuite.html
In short I used several SELECT queries which perform point and range access to table by PrimaryKey and usual key and one query for full table scan.
Range queries is especially interesting for bug 15815 as the same such queries were affected on multi-CPU systems.

Used hardware:

CentOS release 4.4 (Final)
2 Ñ… Dual Core Intel XEON 5130

model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz
stepping : 6
cpu MHz : 1995.004
cache size : 4096 KB

16GB of RAM

Disks do no matter, as I ran clear CPU-bound benchmarks.

The raw results of benchmarks are available here:
http://spreadsheets.google.com/pub?key=pOIo5aX59b6Z8eWjePZU-JQ
or on our page
https://www.percona.com/blog/files/benchmarks/innodb_scale.html
(Note: This benchmark is synthetic micro benchmarks focusing on particular simple data access patterns. Results for your workload are likely to be different.)

Couple of my comments:

• • The test on our hardware did not show the scalability problem, but the main goal was the comparison of
    overall performance of InnoDB before fix (in 5.0.27) and after fix (5.0.30/5.1.14 and later)

• • The fix of 15815 shows good results:
    in 5.0.32 (comparing to 5.0.27) the improvement for READ_PK_RANGE is 5-16%.
    most interesting is improvement by 30-39% in READ_KEY_RANGE queries
    and by 10-23% for full table scan queries.
    I unambiguously recommend to switch to 5.0.30 – 5.0.32 release when it is available.

• • You can see general performance degradation comparing 5.1.14 with 5.0.32
    For example slow down by 12% in READ_PK_POINT with 1 thread.
    And by 11-15% in READ_KEY_RANGE queries.
    There is nothing extraordinary, this is a paid for new features.