Tokutek created the iiBench benchmark back in 2008. The point of the benchmark is to measure the performance of indexed insertions over time. It uses an extremely simple schema, one table with a sequential insertion pattern for the primary key along with three integer fields storing random values. The table maintains 3 secondary indexes, each including several of the random integer fields. The iiBench application itself is currently maintained on Launchpad.
B-tree implementations generally require maintenance operations to update leaf nodes (an insertion is one such operation). When the entire B-tree index does not fit in RAM an IO is required, and performance drops dramatically. Fractal Tree Indexes do not exhibit this performance drop as leaf node maintenance is deferred, allowing for several hundred (or thousand) leaf node maintenance operations to be performed with a single IO.
I created a new version of iiBench specifically for MongoDB to compare the performance over time of TokuMX versus MongoDB. For anyone interested the code is available on GitHub. It is a simple benchmark application written in Java (sorry Henrik) and can easily be modified for other benchmark tasks.
Benchmark Results – Insertion Performance
As the performance graph shows, the insertion performance of MongoDB drops substantially when the indexes no longer fit in RAM while TokuMX does not. The exit throughput* of MongoDB is 169.49 inserts per second, while TokuMX is 10,609.63 inserts per second. In other words, the exit throughput of TokuMX is over 60 times faster than MongoDB.
“exit throughput” = average inserts per second over the last 5 million inserts.
Benchmark Results – IOPs
Also worth noting is a comparison of the IO with MongoDB and TokuMX. Over the entire benchmark, MongoDB averaged 1,378.94 write operations per second, whereas TokuMX averaged just 55.27. Keep in mind that the TokuMX line in this graph is so short since it finished the 100 million inserts in just over 2 hours, MongoDB required over 29 hours for the same number of insertions. Also, I suspect the write IO in MongoDB decreased over time given the reduction of document inserts per second.
Try TokuMX for Yourself
We are in the Release Candidate process for TokuMX, if you are interested in participating please drop us a line.
Percona’s widely read Percona Data Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.
Besides specific database help, the blog also provides notices on upcoming events and webinars.
Want to get weekly updates listing the latest blog posts? Subscribe to our blog now! Submit your email address below and we’ll send you an update every Friday at 1pm ET.