Monitoring the cluster Monitoring the cluster¶
The important bit about the cluster is that each node should be monitored independently. There is no centralized node, the cluster is the set of active, connected nodes, and each can have a different view of the cluster. Further, many of these variables are relative to the node you query them from: for example, replication sent (from this node) and received (from writes on the rest of the cluster). Having data from all nodes helps tracking down the source of issues (i.e., where are the flow control messages coming from? Where did that 100MB transaction came from?).
Manually¶
Manual cluster monitoring can be done with myq_gadgets.
Alerting¶
Standard MySQL alerting should apply here. Percona XtraDB Cluster specific alerting should include:
- Cluster state of each node (
wsrep_cluster_status!= Primary)- Node state (
wsrep_connected,wsrep_ready!= ON)
Other optional alerting could be done on:
- Excessive replication conflicts (high rate of
wsrep_local_cert_failuresandwsrep_local_bf_aborts)- Excessive Flow control messages (
wsrep_flow_control_sent/wsrep_flow_control_recv)- Large replication queues (
wsrep_local_recv_queue).
Metrics¶
Metrics collection (i.e., long-term graphing) on the cluster should be done on:
- Queue sizes (
wsrep_local_recv_queue,wsrep_local_send_queue)- Flow control (
wsrep_flow_control_sent,wsrep_flow_control_recv)- Number of transactions in and out of this node (
wsrep_replicated,wsrep_received)- Number of transactions in and out in bytes (
wsrep_replicated_bytes,wsrep_received_bytes)- Replication conflicts (
wsrep_local_cert_failuresandwsrep_local_bf_aborts)
Other Reading¶
Table Of Contents
- About Percona XtraDB Cluster
- Percona XtraDB Cluster Limitations
- Percona XtraDB Cluster Errata (as of 5.5.34)
- Bootstrapping the cluster
- State Snapshot Transfer
- Xtrabackup SST Configuration
- Restarting the cluster nodes
- Cluster Failover
- Monitoring the cluster
- Installing Percona XtraDB Cluster on CentOS
- Installing Percona XtraDB Cluster on Ubuntu
- Setting up Galera Arbitrator
- How to setup 3 node cluster on single box
- How to setup 3 node cluster in EC2 enviroment
- Load balancing with HAProxy
- Setting up PXC reference architecture with HAProxy
- How to Report Bugs
- Percona XtraDB Cluster Release notes
- Index of wsrep status variables
- Index of wsrep system variables
- Index of
wsrep_provideroptions - Index of files created by PXC
- Frequently Asked Questions
- Glossary