During a typical release cycle for TokuDB at Tokutek, we spend time qualifying and hardening the product using numerous tools. For example, we run stress and unit tests directly on the Fractal Tree indexes, MySQL Test Runner (MTR) tests on the storage engine as well as numerous performance benchmarks to prevent regressions. In addition, we have recently been implementing the Random Query Generator (RQG) framework internally here at Tokutek to more exhaustively stress TokuDB. My name is Joel Epstein and I am a Quality Assurance Engineer here at Tokutek who has been integrating RQG into the overall test plan strategy.
At a high level, RQG is a powerful tool which gives a QA engineer the ability to run thousands of queries generated at random against a randomly generated data set. RQG also has the ability to quickly start a MySQL database with a preconfigured set of mysqld options as well. The framework can be used to induce stress-related crashes and other issues quickly that might not reveal themselves during the course of unit and/or MTR testing. RQG is an open source project that is used by QA organizations such as Percona, MariaDB and Oracle.
During an RQG test run, a MySQL Database can be started, initialized and populated with a pre-configured default set of tables. Once an RQG test run is initiated, all queries generated and executed by the framework are also captured for triage and possible replay should a crash be induced or an issue encountered. RQG also has the ability to spawn concurrent client connections to simulate multi-threaded behavior. This functionality, although it prevents test determinism, can really help stress a storage engine and harden functionality.
RQG can also start, initialize and populate multiple DB instances and be used to compare the results of the same query against two identical databases. Used in this manner, RQG can be a powerful tool to verify correctness as well as stability.
The basic building blocks of an RQG test are a grammar, or .yy file and data, or .zz file. The grammar file contains all keywords and randomized value parameters that are parsed into single queries repeatedly. The data file contains the randomized DB table parameters such as field and data definitions. Together, these two files are the input into the RQG runall.pl script that will rapidly build queries, execute them and validate the output. In many cases, QA test plans can be built from requirements and result in very specific test cases with meticulously tracked expected input and output. RQG does not fall into this category of testing tools. At Tokutek, we currently use it to expand our range of stress-inducing and corruption detection test tools. In my next blog, I will expand upon how we execute TokuDB specific grammar files with TokuDB configuration parameters. Stay tuned…
