Mike is a Senior Staff Engineer on the Slack Infrastructure Engineering team working on hard problems of scalability and reliability and leading the development of Slack's next generation database architecture. Previously, he was co-founder and CTO of Jut -- a startup applying a new dataflow language to observability for developers and operations engineers. Prior to that, he was at Riverbed where he was part of the founding development team for the Steelhead WAN accelerator and later led the Pilot/Shark team building network analytics products based on the wireshark open source packet capture technology. He has a PhD in Computer Science from UC Berkeley and is passionate about large scale distributed systems and performance.
MySQL is the backbone of Slack's data storage infrastructure, handling billions of queries per day across thousands of sharded database hosts. We are the midst of migrating this system to use Vitess' flexible sharding and topology management instead of simple application-based shard routing and manual administration. This effort aims to provide an architecture that scales to meet the growing demands of our largest customers and features while under pressure to maintain a stable and performant service.
This talk will present the core motivations behind our decision, why Vitess won out as the best option, and how we laid the groundwork for the migration within our development teams. We will then present some challenges and surprises (both good and bad) found during our transition and our contributions to the Vitess project that mitigated them. Finally, we will discuss the future plans for our migration and suggest improvements to the Vitess ecosystem to aid other adoption efforts.