Bryana Knight (GitHub) delivers the talk, "Scaling a High-Traffic Database: Moving Tables Across Clusters", on DAY 1 of the Percona Live Open Source Database Conference 2018, 4/24, at Santa Clara, CA.
While GitHub isn't the biggest database around in terms of the amount of data we hold in MySQL, it is among the top 50 busiest sites on the internet. Facing an immediate need to distribute load, we came up with creative ways to move a significant amount of traffic off of our main MySQL cluster, with no user impact. Moving five of our hottest tables required collaboration between engineers, DBAs and SRE. This talk will describe when and how to do it, and prove it to be an efficient database scalability solution.
Moving tables required changes to our database infrastructure as well as our application. I'll explain the impetus for this work and why we did it. We'll walk through the application-level changes that allowed us to change connections while still serving data. Then, I'll discuss the ways we moved tables to different clusters, using MySQL replication, or in some cases, temporary sharding and copying billions of rows. Finally, I'll outline the orchestration of the actual cutovers.
Introduction to pg_stat_monitor
Using PMM to Identify and Troubleshoot Problematic MySQL Queries
How To Measure Linux Performance Wrong
Google Cloud Platform: MySQL at Scale with Reliable HA
Analytical Queries in MySQL - PLO October 2020
MySQL Ecosystem on ARM - Krunal Bauskar - Percona Live ONLINE 2020