From requirements to sharding and partitioning - and everything in between.

Average: 8.6 (5 votes)

Taking a design requirement for a high-volume, high-availability architecture from requirements to finished product

Design is always based on requirements. But what if the requirements are unrealistic? Or requirements don't tell you the whole picture? Or there are requirements that are not obvious? Or advertised features don't work as expected? This talk shows you through a real-life architecture how not to take anything for granted - including requirements, MySQL features and performance.

The design for CLIENTX's troubleshooting and reporting system for their SMS router products completely changed how they think about the product. With a sustained level of up to 60,000 events/sec that needed to be recorded, searched and reported on, it was no unique challenge designing an architecture based on MySQL on commodity hardware. Add to that the requirement that this be capable of scaling all the way from 100 events/sec to 60,000 using the same architecture and you get an idea of the challenge. There was an existing design - the problem was that the way it was implemented, it could not scale beyond 10,000 messages/sec. We had to start from scratch, gathering and analyzing requirements right from the data source to the end-user to come up with a radically new design that not only met the existing criterion but was capable of scaling up almost indefinitely, way beyond the 60,000/sec requirement

I will be talking about how we took on this challenge - and taking nothing for granted, we took this from a pipe-dream to a real, performance architecture that is successfully in use at multiple tier-1 telcos. The final solution uses MySQL on commodity hardware with a combination of intelligent design, partitioning, sharding and aggregation to achieve and sustain these high transaction levels.

Trends in Architecture and Design
Experience level: 

Schedule info

Time slot: 
12 April 13:00 - 13:50
Ballroom B

Schedule Info

12 April 13:00 - 13:50 @
Ballroom B


Senior Consultant, The Pythian Group

Raj is a Consulting DBA with both, a Systems Administration as well as a Development background, which provides a good grip of the fundamentals of any system and allows him to see the big picture. He gained exposure to a number of high-volume, high availability systems through his job as DBA and team lead at the Pythian group, where he was one of the first MySQL DBAs hired 8 years go. Maintaining and tuning those systems gave him the experience to see what works and what doesn't in the field. He is currently consulting with a number of companies to help develop highly available, high-concurrency architectures based on Oracle and MySQL. With a number of successes under his belt with the Web2.0, Media, Telecom, Airline and Healthcare industries, he is well-qualified to address any performance and scaling challenge.


Sponsored By