Everyday We’re Shuffling — Online Shard Migration at Facebook
4 October 3:10 PM - 4:00 PM @ St. Gallen
Experience level:
Beginner
Duration:
50 minutes conference
Tracks:
Scalability/Performance
Architecture/Design
Topics:
MySQL
Optimization
Sharding
Any engineer interested at how we run MySQL at Facebook can tell you that we shard our data, and run multiple shards per MySQL instance. This has been the widely accepted approach to “shared nothing” scaling of MySQL in large installations. What do you do when classic sharding is no longer sufficient? Older shards can grow beyond their expected size, and cause imbalance. Some shards can become too hot, and require more resources to be allocated. In some cases, they can even be underused. These imbalances cause very large inefficiencies in absolute numbers at large scale. Learn how we built our online shard migration solution. We’ll start from the building blocks that explain how we move databases between replica sets, which could be applicable at any scale. Then we will dive into the details of how we constructed an intelligent system which balances capacity and allocates resources in an optimal way, without human intervention.
Speakers
Daren Seagrave
MySQL Infrastructure Engineer, Facebook
Biography:
Daren is a MySQL Infrastructure Engineer at Facebook. Previously worked as Architecture and Data Services Manager at Web Reservations International.