For a long time I’ve wanted to know how MySQL scales as you add more memory to the server. Vadim recently benchmarked the effects of increasing memory and CPU core count. He looked for a balance between utilizing the hardware as much as possible, limiting the system complexity, and lowering the […]Read more
Often times, from a computing perspective, one must run a function on a large amount of input. Often times, the same function must be run on many pieces of input, and this is a very expensive process unless the work can be done in parallel.
Shard-Query introduces set based processing, which on the surface […]
Here is an easy way to run the subset sum check from SQL, which you can then distribute with Shard-Query:
CREATE TABLE `the list` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`val` bigint(20) NOT NULL DEFAULT ‘0’,
PRIMARY KEY (`id`),
KEY `id` (`id`)
SELECT val as `val`, COUNT(DISTINCT (id)) as `cd`
Percona is pleased to welcome Stewart Smith to the team. Stewart does not need an extended introduction for MySQL Community, but just in case: Stewart has a long history with both the MySQL and Drizzle code bases. He’s been one of the core Drizzle developers since the start of the project (working on Drizzle […]Read more
I have one free ticket to give away to RailsConf next week in Baltimore! Post a comment to win, and if you aren’t the winner, I’ll give you a discount code for Percona Live as a consolation prize.
Here’s the catch: you have to find at least one thing wrong with the following typical […]
Demonstrating distributed set processing performance
Shard-Query + ICE scales very well up to at least 20 nodes
This post is a detailed performance analysis of what I’ve coined “distributed set processing”.
Please also read this post’s “sister post” which describes the distributed set processing technique.
Also, remember that Percona can help you get up and running using these […]
There are two ways InnoDB can organize tablespaces. First is when all data, indexes and system buffers are stored in a single tablespace. This is typicaly one or several ibdata files. A well known innodb_file_per_table option brings the second one. Tables and system areas are split into different files. Usually system tablespace is located […]Read more
I’ve recently been blogging about how partitioning is a poor man’s answer to covering indexes. I got the following comment from Jaimie Sirovich:
“There are many environments where you could end up creating N! indices to cover queries for queries against lots of dimensions.”
[Just a note: this is only one of several points he […]
Infobright and InnoDB AMI images are now available
There are now demonstration AMI images for Shard-Query. Each image comes pre-loaded with the data used in the previous Shard-Query blog post. The data in the each image is split into 20 “shards”. This blog post will refer to an EC2 instances as a […]