iiBench Benchmark: TokuMX vs. MongoDB

iiBench Benchmark: TokuMX vs. MongoDB


Tokutek created the iiBench benchmark back in 2008. The point of the benchmark is to measure the performance of indexed insertions over time. It uses an extremely simple schema, one table with a sequential insertion pattern for the primary key along with three integer fields storing random values. The table maintains 3 secondary indexes, each including several of the random integer fields. The iiBench application itself is currently maintained on Launchpad.

B-tree implementations generally require maintenance operations to update leaf nodes (an insertion is one such operation). When the entire B-tree index does not fit in RAM an IO is required, and performance drops dramatically. Fractal Tree Indexes do not exhibit this performance drop as leaf node maintenance is deferred, allowing for several hundred (or thousand) leaf node maintenance operations to be performed with a single IO.

I created a new version of iiBench specifically for MongoDB to compare the performance over time of TokuMX versus MongoDB. For anyone interested the code is available on GitHub. It is a simple benchmark application written in Java (sorry Henrik) and can easily be modified for other benchmark tasks.

Benchmark Environment

  • HP Proliant DL380 G6, (2) Xeon 5520, 16GB RAM, P410i (512MB, write-back), 8x10K SAS/RAID 0
  • Centos 5.8 (64-bit), XFS file system
  • MongoDB v2.2.3 and MongoDB v2.2.0 + Fractal Tree Indexes

Benchmark Results – Insertion Performance

As the performance graph shows, the insertion performance of MongoDB drops substantially when the indexes no longer fit in RAM while TokuMX does not. The exit throughput* of MongoDB is 169.49 inserts per second, while TokuMX is 10,609.63 inserts per second. In other words, the exit throughput of TokuMX is over 60 times faster than MongoDB.

“exit throughput” = average inserts per second over the last 5 million inserts.

Benchmark Results – IOPs

Also worth noting is a comparison of the IO with MongoDB and TokuMX. Over the entire benchmark, MongoDB averaged 1,378.94 write operations per second, whereas TokuMX averaged just 55.27. Keep in mind that the TokuMX line in this graph is so short since it finished the 100 million inserts in just over 2 hours, MongoDB required over 29 hours for the same number of insertions. Also, I suspect the write IO in MongoDB decreased over time given the reduction of document inserts per second.

Try TokuMX for Yourself

We are in the Release Candidate process for TokuMX, if you are interested in participating please drop us a line.


Share this post

Comments (2)

  • Bigger data | InsideMySQL Reply

    […] used the insert benchmark to measure the insert rate, write efficiency and database size. The benchmark loads a […]

    March 12, 2014 at 4:23 am
  • 乱象,印迹 » 翻译文档:TokuMX的分形索引是什么? Reply

    […] 但对TokuMX来说,这是绝对不成立的。依靠TokuMX所提供的创新性的分形树索引,索引的工作集可以比内存更大,同时写入性能不受影响。分形树索引为什么在重度写入(无论是MongoDB还是MySQL)的评测中能表现优异,原因就在这里。 […]

    July 1, 2014 at 9:55 pm

Leave a Reply