Failed ReadTopologyInstanceerror when adding MySQL host to Orchestrator?
The best place to discuss PMM with developers and other community members is the community forum.
If you would like to report a bug, use the PMM project in JIRA.
Any system which can run Docker version 1.12.6 or later.
It needs roughly 1 GB of storage for each monitored database node with data retention set to one week.
Minimum memory is 2 GB for one monitored database node, but it is not linear when you add more nodes. For example, data from 20 nodes should be easily handled with 16 GB.
Any modern 64-bit Linux distribution. It is tested on the latest versions of Debian, Ubuntu, CentOS, and Red Hat Enterprise Linux.
Minimum 100 MB of storage is required for installing the PMM Client package. With good constant connection to PMM Server, additional storage is not required. However, the client needs to store any collected data that it is not able to send over immediately, so additional storage may be required if connection is unstable or throughput is too low.
By default, Prometheus in PMM Server uses up to 768 MB of memory for storing the most recently used data chunks. Depending on the amount of data coming into Prometheus, you may require a higher limit to avoid throttling data ingestion, or allow less memory consumption if it is needed for other processes.
You can control the allowed memory consumption for Prometheus
by passing the
METRICS_MEMORY environment variable
when creating and running the PMM Server container.
To set the environment variable, use the
The value must be passed in kilobytes.
For example, to set the limit to 4 GB of memory:
The limit affects only memory reserved for data chunks. Actual RAM usage by Prometheus is higher. It is recommended to set this limit to roughly 2/3 of the total memory that you are planning to allow for Prometheus. So in the previous example, if you set the limit to 4 GB, then Prometheus will use up to 6 GB of memory.
By default, Prometheus stores time-series data for 30 days, and QAN stores query data for 8 days.
Depending on available disk space and your requirements, you may need to adjust data retention time.
You can control data retention by passing the METRICS_RETENTION and QUERIES_RETENTION environment variables when
creating and running the PMM Server container. To set environment variables, use the
option. The value is passed as a combination of hours, minutes, and
seconds. For example, the default value of 30 days for
720h0m0s. You probably do not need to be
more precise than the number hours, so you can discard the minutes and
seconds. For example, to decrease the retention period for
Prometheus to 8 days:
When you add a monitoring instance using the pmm-admin tool,
it creates a corresponding service.
The name of the service has the following syntax:
The location of the services depends on the service manager:
|Service manager||Service location|
To see which service manager is used on your system, run as root pmm-admin info.
Every service created by pmm-admin when you add a monitoring instance gets a
DSN from the credentials provided, auto-detected, or created (when adding the
instance with the
For MySQL and MongoDB metrics instances (
mongodb:metrics services), the DSN is stored with the corresponding
service files. For more information, see Where are the services created by PMM Client?.
For QAN instances (
mysql:queries service), the DSN is stored in local
configuration files under
Also, a sanitized copy of DSN (without the passowrd) is stored in Consul API for information purposes (used by the pmm-admin list command).
Every service created by pmm-admin when you add a monitoring instance has a
separate log file located in
/var/log/. The file names have the
For example, the log file for the MySQL QAN monitoring service is
You can view all available monitoring instance types and corresponding ports using the pmm-admin list command. For more information, see Listing monitoring services.
PMM Server runs
logrotate to rotate nginx logs on a daily basis
and keep up to 10 latest log files.
If a MySQL server has a lot of schemas or tables, it is recommended to disable per table metrics when adding the instance:
sudo pmm-admin add mysql --disable-tablestats
Table statistics are disabled automatically if there are over 1 000 tables.
For more information, run as root
Yes, you can use pmm-admin to start and stop either individual services that correspond to the added monitoring instances, or all of them at once.
To stop all services:
sudo pmm-admin stop --all
To start all services:
sudo pmm-admin start --all
For more information about starting and stopping services, see Starting monitoring services.
You can view all available monitoring instances and the states of the corresponding services using the pmm-admin list command. For more information, see Listing monitoring services.
When adding MySQL instance to monitoring,
you can specify the MySQL server superuser account credentials,
which has all privileges.
However, monitoring with the superuser account is not secure.
If you also specify the
it will create a user with only the necessary privileges for collecting data.
You can also set up the
pmm user manually with necessary privileges
and pass its credentials when adding the instance.
To enable complete MySQL instance monitoring, a command similar to the following is recommended:
sudo pmm-admin add mysql --user root --password root --create-user
The superuser credentials are required only to set up the
pmm user with
necessary privileges for collecting data. If you want to create this user
yourself, the following privileges are required:
GRANT SELECT, PROCESS, SUPER, REPLICATION CLIENT, RELOAD ON *.* TO 'pmm'@' localhost' IDENTIFIED BY 'pass' WITH MAX_USER_CONNECTIONS 10; GRANT SELECT, UPDATE, DELETE, DROP ON performance_schema.* TO 'pmm'@' localhost';
pmm user already exists,
simply pass its credential when you add the instance:
sudo pmm-admin add mysql --user pmm --password pass
For more information, run as root
Yes, you can add multiple MySQL instances to be monitored from one
PMM Client. In this case, you will need to provide a distinct port and socket
for each instance using the
--socket parameters, and specify
a unique name for each instance (by default, it uses the name of the
PMM Client host).
For example, if you are adding complete MySQL monitoring for two local MySQL servers, the commands could look similar to the following:
$ sudo pmm-admin add mysql --user root --password root --create-user --port 3001 instance-01 $ sudo pmm-admin add mysql --user root --password root --create-user --port 3002 instance-02
For more information, run
You can remove any monitoring instance as described in Removing monitoring services and then add it back with a different name.
When you remove a monitoring service, previously collected data remains available in Grafana. However, the metrics are tied to the instance name. So if you add the same instance back with a different name, it will be considered a new instance with a new set of metrics. So if you are re-adding an instance and want to keep its previous data, add it with the same name.
When you add an instance with the pmm-admin tool, it creates a corresponding service that listens on a predefined client port:
|General OS metrics||
If a default port for the service is not available, pmm-admin automatically chooses a different port.
If you want to assign a different port, use the
when adding instances.
When using Docker, the container is constrained by the host-level routing and firewall rules. For example, your hosting provider might have default iptables rules on their hosts that block communication between PMM Server and PMM Client, resulting in DOWN targets in Prometheus. If this happens, check firewall and routing settings on the Docker host.
If you encounter communication issues, try the following:
netstaton PMM Client to see what state the connections are in
curl www.google.comto see if you get a reply
mysql:metrics service collects metrics with different resolutions (1
second, 5 seconds, and 60 seconds)
mongodb:metrics services are set up to collect
metrics with 1 second resolution.
In case of bad network connectivity between PMM Server and PMM Client or
between PMM Client and the database server it is monitoring, scraping every
second may not be possible when latency is higher than 1 second. You can change
the minimum resolution for metrics by passing the
environment variable when creating and running the PMM Server container. To set this environment variable, use the
The values can be between 1s (default) and 5s. If you set a higher value,
Prometheus will not start.
For example, to set the minimum resolution to 3 seconds:
Consider increasing minimum resolution when PMM Server and PMM Client are on different networks, or when Using PMM with Amazon RDS.
You need to create Orchestrator’s topology user on MySQL according to this section.
For general inquiries, please send us your question and someone will contact you.