This checklist will help make the most of your time with Percona by ensuring our access to your servers is correctly set up before we begin working with you.
Please make sure you've provided us with the items listed so we can log in to your servers. If there's more than one server, please tell us each of the servers’ function (e.g., “master” or “read-only slave”), and all other information that applies to each server. This helps ensure we are working on the intended servers. For each server, set up a “message of the day” (MotD) indicating the server's function. Be sure to point out critical things we should be careful not to disturb.
Engineer Keys: The New Method
As of Q1 2016, engineer public SSH keys are available from the percona.com domain. For example:
We provide two types of engineer packages: one individual that creates a user for the engineer and adds its public key to the user, and a global package that ships only the public keys (it doesn’t create users).
What Engineer individual packages do:
- Creates /home/firstname.lastname
- Installs pubkeys into /home/firstname.lastname/.ssh/authorized_keys
The shared RPMS, however installs just the keys. For example:
Installs all engineer pubkeys into /usr/share/percona/engineer/keys
This package installs all engineer pubkeys to /usr/share/percona/engineer/keys/firstname.lastname.pub.
No user management is performed, the files are used only for reference.
As an example, to add one engineer key to shared Percona users:
cat /usr/share/percona/engineer/*.pub >
Please note that old keys are not removed or managed in any way by this method. You must remove them manually.
Both packages are created every 12 hours. RPMS are not GPG signed at this time. This is to ease clients' configuration and maintenance during their engagement with Percona.
Engineers connect from service bastions with public IPs 184.108.40.206 and 220.127.116.11.
Legacy Keys: The Old Method
This method is still valid, but we invite customers to install engineers keys instead of the shared key.
OS Percona User
Create a user named percona, install our SSH key in percona's account and grant sudo with no password.
You can permit SSH access with this key by placing it in .ssh/authorized_keys. You can run the following commands to install the SSH key. Log in as the user you want to grant access before running them:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
cat << EOF >> ~/.ssh/authorized_keys
The commands above should work correctly. Please be sure you tell us the username we should use to log in.
Please check that:
- The key is installed in the correct home directory
- The key set to 600 permissions
- The key no extra line breaks
- The user you've specified is in the SSH configuration's permitted user or permitted group list (if that is restricted)
You can also download the key from this link.
Engineers connect from this two IPs:
- bm01.jh.percona.com (18.104.22.168)
- bm02.jh.percona.com (22.214.171.124)
Note. We will not use those IP address or SSH key if you require VPN access. If you use VPN, talk to your consultant about how to set up access.
MySQL Percona User
Create a MySQL user named percona, grant all privileges on *.* and give us the password (or leave it in a file in Percona's home directory).
Extended information about the required MySQL account on your database server:
- What is the username and password of the MySQL account we should use?
- Have you granted SELECT, SUPER, PROCESS, RELOAD ON *.* to the user?
- What is the IP address or hostname of the MySQL server?
- On what IP address does MySQL listen, if bind_address is set in its configuration?
- What is the port or socket, if they are non-standard?
- If possible, create a “percona” database and give us all privileges to it (to create temporary tables while optimizing queries). This is completely optional, but it's helpful to us. You can do it by running: CREATE DATABASE percona; GRANT ALL ON percona.* TO 'user'@'host';
When you exchange information with us, or give us access to your servers, please make sure you do it securely. If you need to upload sensitive data to our servers securely for data recovery purposes, please request our GPG key. This lets you encrypt large files that only we can decrypt.
For security and accountability reasons, please do not do any of the following:
- Don't permit Percona to log into a shared account. Create a dedicated Percona user.
- Don't install our SSH key in the root's home directory.
- Don't permit Percona to use a common MySQL account such as root. Create a dedicated Percona user in MySQL.
- Don't permit our SSH key to be forwarded through an agent to another host. If we must access your systems through an intermediary "bastion" server and please create an SSH key on that server. Install the matching the public key on the systems we should be able to access.
Other Types of Access
We can work with nearly any type of access (or even none at all). Direct SSH access to the server is most efficient, however, and any more elaborate types of access (such as proprietary VPN programs, platform-dependent technologies such as GoToMyPC, etc.,) can be less efficient -- sometimes significantly less efficient.
Many of these application cannot coexist, so they require elaborate tricks with virtual machines, etc. Multiple hops are also less efficient, as keystroke latency becomes a problem. The use of a graphical desktop on a remote system is also difficult.
In general, Cisco VPN and OpenVPN are the most efficient VPNs to use. If you're using VPN, please tell us all the necessary information:
- What type of VPN system are we connecting to? Do we need to download a client?
- Please send us the .PCF file (if you're using a Cisco VPN)
- Otherwise, please tell us all of the following that apply: the domain server's hostname and IP address, username, password, group name, group password
- VPN might not work well with DNS, so please tell us the IP addresses for the servers
If you're using any other type of connectivity, such as VNC, Remote Desktop, GoToMyPC, or similar, please provide the details.
The most common reason we can't log in is that the installed SSH key is for a different user than the one Percona was assigned. The next most common reason is that there are line breaks, incorrect permissions, or other problems with the key file. After you've set the key up, please let us know so we can test it.
Note that VPN and our shared SSH key do not play well together. For security reasons, consultants do not have the passphrase to our shared SSH key, so they are unable to load it on their own computers. We cannot use VPN on our shared login host, which we use for SSH-keyed access. If you are using VPN, consultants will log in from their own machines, so do not restrict access to any specific range of IP addresses. Also, be sure to provide password authentication or an SSH key of your choosing.
Performance Audit Checklist
A performance audit is our most popular service. In order to expedite a performance audit, please ensure that the following tools are installed on each machine:
- Perl DBI and DBD::mysql
- vmstat, iostat, and optionally innotop, mpstat, and dstat
- If there is no outside network access, Percona Toolkit
If you can't install any one for some reason, we can still perform an audit. But these tools are very helpful for observing what the machine is doing.
Part of our standard procedure is to review the MySQL slow query logs. Having them enabled before we begin can save time. If your slow query log is not enabled, you can enable it by adding the following lines to my.cnf.
Part of our standard procedure is to review the MySQL slow query logs. Having them enabled before we begin can save time. If your slow query log is not enabled, you can enable it by adding the following lines to my.cnf. This will require a MySQL restart to take effect in standard MySQL 5.0 and earlier. Please check the MySQL error log and ensure that it contains no errors after the restart; if the slow query log file can't be created for some reason, the error log will alert you to this fact.
log-slow-queries = /path/to/file/slow.log
long_query_time = 1000
This will require a MySQL restart to take effect in standard MySQL 5.0 and earlier. Please check the MySQL error log and ensure that it contains no errors after the restart. If the slow query log file can't be created for some reason, the error log will alert you to this fact.
A short architectural overview can go a long way towards giving us a “head start” on the performance audit. Please provide brief documentation that includes:
- Language(s) used
- Deployment diagrams (if any)
- Components used (load balancers, caching systems, etc...)
- Database replication/sharding Information
- Planned upgrades/changes – alerting us in advance of any new features or changes planned during the time of the audit can save significant time and rework
- Anything else you feel is worth mentioning
You don't have to go to extremes with any of this. Too much information can cause extra work, too (lists of tables and columns are not necessary). Keep in mind that we have probably seen dozens of systems like yours, so just giving us “hints” about what's unique or noteworthy about your systems can be very effective. For example, “it's a social networking application using Ruby On Rails and we are not sharded yet, but we do a lot of caching” tells us a great deal.
We highly recommend that all of our customers have alerting and monitoring systems in place. Although there are many of these packages available, we have found the combination of Nagios and Cacti to be very effective. Nagios MySQL plugins are available from www.nagiosexchange.com
and MySQL Cacti templates can be downloaded from here
If you need help installing or configuring them, please let us know. We are glad to assist.
Amazon RDS - Relational Database Service
Ideally you will provide to Percona remote SSH access to a secured Linux server in your own controlled VPC subnet in AWS. You will securely (by way of GPG or other encryption medium) provide a MySQL account for Percona against your RDS instance so that all communication between MySQL client and RDS happens on Amazon's private internal network. Any auditing or compliance requirements are to be made known to Percona prior to engagement.