Speeding Up TPCC Table Loads by 8x with TokuDB v5.0

Posted on:



Share Button

Percona’s TPCC for MySQL toolset allows one to measure the query performance for an OLTP workload on various MySQL storage engines. The toolset includes a program to load the database tables, and a program to run queries and measure performance. We have found Percona’s TPCC toolset to be extremely useful for tuning our software. However, we want to take advantage of TokuDB’s bulk load capability when loading the database.

We created a new tool, a simple variant of the existing code, that generates CSV files for the TPCC database. These CSV files can be bulk loaded into TokuDB with a “LOAD DATA INFILE” statement. TokuDB’s bulk loader uses a parallel merge sort algorithm that is implemented in CILK, an extension to the C language that allows one to easily exploit multiple core machines. In contrast, Percona’s TPCC loader inserts one row at a time, and typically uses one core.

We ran TPCC experiments on an Amazon EC2 machine (c1.xlarge, 8 cores, 7 GB of memory) with a single EBS volume for storage. All experiments used the latest version of TokuDB release 5.0. We observed an 8 fold speedup on an 8 core machine when loading the 200 warehouse TPCC database into TokuDB using the bulk loader, compared with Percona’s tpcc_load program. We expect further performance increases on machines with additional cores.

Share Button

Tokutek, TokuView

  • Are queries multithreaded as well? I read something that it wasn’t in the initial release.

  • Rich Prohaska Post author

    The performance on multi-thread workloads like TPCC and sysbench was significantly increased in TokuDB 6.1.0 compared to prior releases.

Leave a Reply