Are you dealing with the challenges of rapid growth? Are you thinking about how to scale your database layer? Should you use NoSQL? Should you shard your relational database? If you are facing these kinds of problems, this tutorial is for you.
Vitess is a database solution for deploying, scaling and managing large clusters of MySQL instances. It's architected to run as effectively in a public or private cloud architecture as it does on dedicated hardware. It combines and extends many important MySQL features with the scalability of a NoSQL database.
By the end of the session, attendees will understand the basics of how to get a Vitess cluster up and running. They will also understand how to perform most of the typical operations you face while running Vitess clusters in a production environment.
This is a hands-on tutorial covers how to set up monitoring for MySQL database servers using the Percona Monitoring and Management (PMM) platform.
PMM is an open-source collection of tools for managing and monitoring MySQL and MongoDB performance. It provides thorough time-based analysis for database servers to ensure that they work as efficiently as possible.
You will learn about:
- MySQL monitoring best practices
- Metrics and time series
- Data collection, management and visualization tools
- Monitoring deployment
- How to use graphs to spot performance issues
- Query analytics
- Trending and capacity planning
- How to monitor HA
Please bring a laptop with the Virtualbox application.
The interaction between applications and the database is one of the most intricate and important in the systems we build. This boundary region is special and complex for a number of reasons. It's the surface that separates stateful from stateless, off-the-shelf from custom, and battle-tested atoms from fast-changing molecules. But more than that, it's not crisply defined. There's no bright line that divides these worlds. Instead, there's a zone where each region is partially enmeshed. Consider schema and indexing design, for example, which is a combination of predefined and user-defined; the query language and its interplay with the planner is another. This is a zone of complexity and richness, where lots of gnarly things sprout, but also where there's a lot of opportunity to learn, so you can design and improve applications better. Baron will share lessons learned from his career of observing how applications and databases interact. You'll leave with insights that will help you see new ways to build better.
Percona Monitoring and Management is quickly growing in both functionality and adoption. In this talk, we will discuss how to integrate it with other Database Management Systems, to get the best out of it on environments that not only use MySQL- or MongoDB-based solutions.
We will focus on how to set up monitoring for PostgreSQL and ClickHouse.
The presentation will be a real-life study on how we use PMM for monitoring of 120+ MySQL and ProxySQL-servers, as well as query optimisation
During the project, we found a few caveats, that others embarking this journey should be aware of. We've also found a few "hidden features", where we're able to use PMM in ways beyond the standard interfaces, due to the fact that it's all built on open and battle-tested software.
While GitHub isn't the biggest database around in terms of the amount of data we hold in MySQL, it is among the top 50 busiest sites on the internet. Facing an immediate need to distribute load, we came up with creative ways to move a significant amount of traffic off of our main MySQL cluster, with no user impact. Moving five of our hottest tables required collaboration between engineers, DBAs and SRE. This talk will describe when and how to do it, and prove it to be an efficient database scalability solution.
Moving tables required changes to our database infrastructure as well as our application. I'll explain the impetus for this work and why we did it. We'll walk through the application-level changes that allowed us to change connections while still serving data. Then, I'll discuss the ways we moved tables to different clusters, using MySQL replication, or in some cases, temporary sharding and copying billions of rows. Finally, I'll outline the orchestration of the actual cutovers.
Time series databases are sprouting up like mushrooms. At Grafana Labs, the company behind Grafana, we built a new engine specifically for GrafanaCloud. Why would we do that? Learn about the design considerations, lessons learned, and tradeoffs we made in designing this engine that is compatible with both Graphite and Prometheus.
Query tuning can be complex. It's often hard to know which knob to turn or button to press to get the biggest performance boost. In this presentation, Janis Griffin, database performance evangelist at SolarWinds, will share her secrets for determining the best approach for tuning queries by utilizing the performance schema (specifically instrumented wait events and thread states), query execution plans, SQL diagramming techniques and more.
Regardless of the complexity of your database or your skill level, this systematic approach will lead you down the correct tuning path with no guessing, saving countless hours of tuning queries and optimizing performance of your MySQL® databases.
- Learn how to effectively use the performance schema to quickly identify bottlenecks and get clues on the best tuning approach
- Quickly identify inefficient operations through review of query execution plans
- Learn how to use SQL diagramming techniques to find the best plan
At Yelp we have a constantly growing polyglot data tier consisting of datastores such as Cassandra, Elasticsearch, MySQL and Zookeeper. These distributed datastores often ask to be treated like pets but can only be reared like cattle given the scale of our systems. Requiring engineers to pamper them individually is neither feasible nor scalable. We need cluster automation which is powerful, resilient and reliable, and more importantly safe. This is where Taskerman steps in.
Taskerman is a distributed cluster task manager, wearing many hats to keep our clusters highly available, consistent, secure and in an optimal condition. Reusability has also been our focus, hence Taskerman has been built on top of AWS and existing open source infrastructures like Yelp PaaSTA, Zookeeper and Sensu.
This talk covers the genesis of Taskerman inside Yelp, its architecture and evolution. Much like the infrastructure it stands on top of, we also hope to open-source Taskerman in the future.
Optimizing MySQL performance and troubleshooting MySQL problems are two of the most critical and challenging tasks for MySQL DBA's. The databases powering your applications need to be able to handle heavy traffic loads while remaining responsive and stable so that you can deliver an excellent user experience. Further, DBA's are also expected to find cost-efficient means of solving these issues.In this presentation, we will discuss how you can optimize and troubleshoot MySQL performance and demonstrate how Percona Monitoring and Management (PMM) enables you to solve these challenges using free and open source software. We will look at specific, common MySQL problems and review the essential components in PMM that allow you to diagnose and resolve them.
In this talk we will review the new functionality released by Amazon Web Services, that allows us to import data from our non-RDS MySQL instances, to RDS instances (MySQL or Aurora). We'll see what works, what doesn't, and how to do it.
Database backup and validation is a trending topic these years. In order to survive from all the possible accidents, Facebook designed and implemented a MySQL backup and validation system for large scale deployment. In this talk, I'll share the implementation detail for this system and the evolution story of it.