InnoDB, InnoDB-plugin vs XtraDB on fast storage

To continue fun with FusionIO cards, I wanted to check how MySQL / InnoDB performs here. For benchmark I took MySQL 5.1.42 with built-in InnoDB, InnoDB-plugin 1.0.6, and XtraDB 1.0.6-9 ( InnoDB with Percona patches).
As benchmark engine I used tpcc-mysql with 1000 warehouses ( which gives around 90GB of data + indexes) on my workhourse Dell PowerEdge R900 ( details about box ).

On storage configuration: FusionIO 160GB SLC and 320GB MLC cards are configured in software RAID0 to store InnoDB datafiles. For InnoDB logs and system tablespace I used partition on regular RAID10 with regular hard drives, here I followed Yoshinori Matsunobu’s recommendations and taking fact that FusionIO is not perfect for sequential writes

Full results I put on page, here are my thoughts and interesting facts.

First, chart with results for InnoDB vs InnoDB-plugin during runs (values are in new order transactions per minute, more is better) :


As you see InnoDB-plugin is doing much better here, it allows to utilize multiple IO threads,
which as we saw ( ) is necessary to get most throughput from FusionIO.

Also you may see from graph some waves for InnoDB-plugin. Here we observe innodb_adaptive_flushing in action (which is ON by default), and I think innodb_adaptive_flushing in InnoDB-plugin is not quite balanced, it may do overaggressive flushing, when it is not necessary.

But looking on CPU stats (see graph later), I guess InnoDB-plugin spends most time in buffer_pool mutex, contention here is not fully resolved yet in InnoDB-plugin.

Now, let’s take XtraDB. In additional to multiple IO threads, we have patch to decrease contention on buffer_pool mutex, plus separate purge thread. Also we use different adaptive_checkpoint algorithm.

The results are:

So I guess buffer_pool improvements play here for XtraDB, and looking on summary result:

  • InnoDB 9439.316 NOTPM
  • InnoDB-plugin-1.0.6 15299.333 NOTPM
  • XtraDB-1.0.6-9 26160.551 NOTPM

InnoDB-plugin is 1.6x times better InnoDB, and XtraDB is 1.7x times better InnoDB-plugin.

Now on CPU usage and disk utilization.

Disk throughput:


CPU (user) usage:


Even with improvements, XtraDB performs less 150MB/s in disk writes (from benchmarks we
saw FusionIO can do much more) and with 45-50% of idle CPU.

I assume we still see significant contentions inside XtraDB, and there big room for improvements. As for InnoDB-plugin, I’d wish InnoDB team makes some actions on buffer_pool mutex problem.

Finally I wanted to check what if we put innodb transactional logs and system tablespace on FusionIO also, there is graph for that:


It is not so bad, with final result 23038.283 NOTPM, it is only about 12% worse than with logs on separate partition.

And to make reference point, I run the same but with all files on RAID10 with regular disks,
the graph is there:


with final result: 2873.783 NOTPM ( about 88% worse than all files on FusionIO)

To summarize

  • MySQL InnoDB/InnoDB-plugin/XtraDB is not fully able to utilize throughput of FusionIO. XtraDB is doing better job with internal contention, but much more can be done.
  • Still you can have very impressive performance improvement in IO-intensive or IO-bound workloads. You may want to use InnoDB-plugin or XtraDB to get better results.
  • Putting logs on separate partition may be good idea, but only if you have possibility to do that. Making special setup for that may be not worth improvements

Share this post

Comments (13)

  • Magento

    It’s a difficult choices between XtraDB innodb & innodb plugin.

    January 13, 2010 at 12:00 am
  • dspezia

    Very interesting. Thanks for sharing this benchmark.
    Which I/O elevator did you use for the OS? Noop?

    January 13, 2010 at 6:29 am
  • Sadao

    tpcc-mysql shows the number of transactions processed every 10 seconds.
    So in these graphes, y-axis stands for not ‘NOTPM’ but ‘NOTP10SEC’ 😉

    January 13, 2010 at 7:43 am
  • Vadim


    Usually I used deadline (for RAID), but
    in this case there is no scheduler settings for software RAID nor for FusionIO cards.

    January 13, 2010 at 9:42 am
  • Vadim


    Thank you, I fixed that.

    January 13, 2010 at 9:42 am
  • Justin Swanhart

    Do you think you would see any differences with the adaptive hash indexes disabled?

    January 13, 2010 at 8:58 pm
  • David

    I think it is problem of adaptive flushing algorithms
    It’s not so aggressive to use the io
    set innodb_io_capacity=500 and disable the adaptive flushing to see the result

    January 13, 2010 at 9:07 pm
  • Vadim


    I just made another run with the adaptive hash indexes disabled, but it does not help, and
    final result is worse.

    Actually in most cases, despite there is contention on adaptive hash index,
    disabling it does not help, as in this case, InnoDB has to do much more work on index pages scan.

    January 14, 2010 at 11:47 am
  • Vojtech Kurka


    did you measure query latency? Throughput benchmarks look great, but the latency is crucial for web applications. We’re using XtraDB 7 with 40GB buffer pool, RAID10 with HDDs and when the write throughput goes up, we see read query latency dips in about 500-1500ms.

    What about the XtraDB RAID10 graph – the first part of the graph looks scary. Is there any explanation for that.

    Thank you for your great job and for publishing these benchmarks 🙂

    January 15, 2010 at 8:09 am
  • Vadim


    Unfortunately latency report from tpcc-mysql benchmark is rather limited, we may fix that one day.

    I have response time histograms for XtraDB and InnoDB-plugin



    (time,sec) (count of transactions)
    0.20, 1658223
    0.40, 2095
    0.60, 107
    0.80, 32
    1.00, 32


    0.20, 945959
    0.40, 6285
    0.60, 319
    0.80, 56
    1.00, 23

    It’s expected that increased throughput affects response time, from queue theory,
    the response time grow exponentially on increasing throughput.

    First part on RAID10 graph is just buffer_pool is warming up. I show results
    from 0 point of time (from restart of mysqld), and for RAID10 HDD it takes much longer to fill buffer_pool.

    January 15, 2010 at 7:25 pm
  • Vojtech Kurka

    “First part on RAID10 graph is just buffer_pool is warming up.”

    I know, but I expected the graph to be much more linear. From this graph it seems the throughput touches zero all the time, until some volume of data gets to the buffer pool.

    January 19, 2010 at 11:45 am
  • Ben

    I’ve actually found xtradb SLOWER than innodb on my own benchmarks.

    I wonder if I’ve missed something with configuring Percona server?

    November 20, 2010 at 8:56 am
  • Satya


    I did load test between Percona server and MySQL server using msqlslap

    mysqlslap -uroot -pimi@123 –query=”INSERT INTO sbtest3.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()));INSERT INTO sbtest2.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()));INSERT INTO sbtest3.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()));delete from sbtest2.sbtest where a like ‘a%’;INSERT INTO sbtest2.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND())); delete from sbtest2.sbtest where a like ‘b%’;INSERT INTO sbtest3.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()));update sbtest2.sbtest set b=’imi’;update sbtest3.sbtest set c=’imi’;INSERT INTO sbtest2.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()));INSERT INTO sbtest3.sbtest VALUES(MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()),MD5(RAND()))” –concurrency=50 –iterations=100

    Table structure of sbtest2.sbtest is
    CREATE TABLE `sbtest` (
    `a` varchar(100) DEFAULT NULL,
    `b` varchar(100) DEFAULT NULL,
    `c` varchar(100) DEFAULT NULL,
    `d` varchar(100) DEFAULT NULL,
    `e` varchar(100) DEFAULT NULL

    Table structure of sbtest3.sbtest is
    CREATE TABLE `sbtest` (
    `a` varchar(100) DEFAULT NULL,
    `b` varchar(100) DEFAULT NULL,
    `c` varchar(100) DEFAULT NULL,
    `d` varchar(100) DEFAULT NULL,
    `e` varchar(100) DEFAULT NULL

    In this case, percona server takes more time than MySQL server.

    Percona : 9:35 min
    MySQL : 8:02 min

    Is there any problem in my load test?
    Please suggest me.

    January 23, 2013 at 2:36 am

Comments are closed.

Use Percona's Technical Forum to ask any follow-up questions on this blog topic.