Understanding the queues your query waits in
Queues are essential to software systems, present everywhere from MySQL to RollerCoaster Tycoon. Understanding these queues is key to achieving consistently high performance. In this session, we will consider many of queues involved in servicing database queries, and some of the queueing theory we can use to optimize them. Using examples from MySQL and Cassandra, the session will examine the layers of a database system from network to compute to storage, and learn about the queues involved in each layer. We will discuss utilities for monitoring queueing at all layers, including the infamous load average and the hard-to-parse iostat. Finally, we will explore the multi-tenancy inherent in every database system, focusing on the interaction between latency-critical processes like queries and batch processes like log writeback, compaction, and backup. Using our knowledge of queueing, we will develop a detailed understanding of tools for managing the performance of all of these tasks.
Site Reliability Engineer, Fitbit
Josh is a Database Engineer at Fitbit. A veteran infrastructurer, he has been known to work on everything from load-balancers to deployment to metrics collection. His biggest focus is stateful services, including datastores of both the SQL (MySQL, Postgres) and NoSQL (Cassandra, Consul) variety. Recently he’s been spending a lot of time with Cassandra: scaling, monitoring, and automating Fitbit’s clusters. Strike up a conversation with him about methods for securing Postgres authentication, restoring backups extremely quickly, or providing a consistent view of a replicated MySQL cluster.