Percona Live: Open Source Database Performance Conference - Amsterdam 2016 Logo

October 3-5, 2016

Amsterdam, Netherlands

Everyday We’re Shuffling — Online Shard Migration at Facebook

Everyday We’re Shuffling — Online Shard Migration at Facebook

 4 October 3:10 PM - 4:00 PM @ St. Gallen
Experience level: 
50 minutes conference


Any engineer interested at how we run MySQL at Facebook can tell you that we shard our data, and run multiple shards per MySQL instance. This has been the widely accepted approach to “shared nothing” scaling of MySQL in large installations. What do you do when classic sharding is no longer sufficient? Older shards can grow beyond their expected size, and cause imbalance. Some shards can become too hot, and require more resources to be allocated. In some cases, they can even be underused. These imbalances cause very large inefficiencies in absolute numbers at large scale. Learn how we built our online shard migration solution. We’ll start from the building blocks that explain how we move databases between replica sets, which could be applicable at any scale. Then we will dive into the details of how we constructed an intelligent system which balances capacity and allocates resources in an optimal way, without human intervention.


Daren Seagrave's picture

Daren Seagrave

MySQL Infrastructure Engineer, Facebook


Daren is a MySQL Infrastructure Engineer at Facebook. Previously worked as Architecture and Data Services Manager at Web Reservations International.

Share this talk