The Big Data Opportunity
“What if you could analyze every transaction, capture insights from every customer interaction, and didn’t have to wait for months to get data from the field?”- McKinsey Column, Financial Times
Despite this tremendous opportunity, MySQL implementations are usually kept relatively small, often on the order of 500 GB or less. Anything lager than this threshold tends to lead to big operational problems, i.e. poor insertion rates, slow queries, hours to days for index rebuilding, offline schema changes, long loading times, dump/reload cycles during down time, extended recovery, etc. The promise of Big Data has remained largely unfulfilled for MySQL. Until now.
Live from NY, at the GigaOM Structure Big Data 2011 event, Tokutek is pleased to announce TokuDB v5.0 for MySQL and MariaDB. TokuDB v5.0 is a powerful and agile storage engine that breaks through MySQL scalability and performance barriers.
Real world data models are constantly changing with market needs. With Hot Column Addition and Hot Indexing, a MySQL schema no longer has to be “set in stone” based on multiple protracted conversations and planning sessions between business analysts and DBAs. Adding a new index to do testing, or adding fields to support better analytics, should always have been a painless value-added processes rather than a disruptive one with downtime – and now it finally is. This “zero maintenance downtime” approach carries over from previous generations of the product. Fractal Tree indexes have always been fragmentation-free, avoiding extended downtime for dump and reload operations.
TokuDB v5.0 continues to be the ideal choice for complex / high-volume applications that must have fast response times and that must simultaneously store and query large volumes of rapidly arriving data. These include social networking, real-time clickstream analysis, logfile analysis, eCommerce Personalization and high-speed webcrawling. TokuDB has demonstrated insertion performance improvements as high as 80x in actual field conditions as well as accelerated queries through better indexing. In addition TokuDB is immune to query slow downs from database aging, and offers a multi-threaded Fast Loader for dramatically reduced load time (1M rows/second or more depending on core count).
Customer deployments for TokuDB start as small as tens of GBs and range all the way up to deployments of multiple TBs. With improved performance at scale (tens of billions of rows), large deployments can easily be handled without resorting to partition based workarounds that are difficult to maintain. The technology insures queries takes full advantage of available disk bandwidth for hard drives (HDD) and maximizes cache utilization for flash drives (SSD). In addition, many customers benefit from compression rates of 5x-15x, greatly reducing additional hardware expenditures.
Stay tuned. In the next few days we will post more detailed blogs on all of the new features. In the meantime please download a free trial. For more details on TokuDB v5.0, please visit the product page or take a look at our overview presentation. You can also go visit our newly created Tokutek YouTube channel where we have recently posted a short video on how Fractal Tree Indexing works. Also, watch out for the MySQL Database Community Podcast this Friday (3/25), where Martín Farach-Colton, our CTO, will talk about TokuDB v5.0 (Episode #39).