What’s New in Percona Toolkit 3.6.0

Percona Toolkit 3.6.0 has been released on June 12, 2024. The most important updates in this version are:

• The possibility to resume pt-online-schema-change if it is interrupted.

• eu-stack support in pt-pmp that significantly improves this tool’s performance and decreases the load it causes on production servers.

• New tool pt-eustack-resolver

• Packages for Ubuntu 24.04 (Noble Numbat)

Other changes include:

• Improvements in pt-galera-log-explainer, pt-k8s-debug-collector, pt-show-grants, pt-summary.

• Bug fixes.

In this blog, I will outline the most significant changes. A full list of improvements and bug fixes can be found in the release notes.

pt-online-schema-change

The tool has the following new options:

• –where that instructs pt-online-schema-change to copy only rows matching specified WHERE clause.
• –history to write progress into the history table.
• –history-table in case you want to override the default name percona.pt_osc_history.
• –resume allows you to resume previously failed jobs stored in the history table.
• –binary-index modifies the behavior of –history such that the history table’s upper and lower boundary columns are created with the BLOB data type. This option works similarly to option –binary-index in the pt-table-checksum.

In the next few weeks, we will release a blog post about resume support in pt-online-schema-change.

In this version, we also fixed PT-1911 – pt-online-schema-change can’t use options port. Now, the option –port is supported the same way as in other tools.

pt-show-grants

We started working on better compatibility with MySQL 8.0 and newer versions. One of the changes is to use option –print_identified_with_as_hex when connecting to a MySQL server that supports this option. This allows the display of hashes for passwords that use caching_sha2_password and sha256_password authentication plugins.

The tool also does not exclude creating an additional CREATE USER statement when dumping grants of the MariaDB server, so such grants could be imported into MySQL 8.0, which does not support user passwords in the GRANT statements. The fix was contributed by Marco Tusa.

pt-pmp

pt-pmp is a profiler tool that creates and summarizes full stack traces of processes on Linux. pt-pmp was inspired by http://poormansprofiler.org and has helped Percona’s Support team resolve many performance issues.

In version 3.6.0, we added an improved pt-pmp port with eu-stack support, contributed by Alexey Stroganov. This option is available via a new option: –dumper (-d). It could be set either to gdb (default), eu (eu-stack), or pteu (pt-eustack-resolver).

eu-stack, unlike gdb, does not cause stalls in the running process it is applied to, so using pt-pmp with eu-stack is much safer in production. Additionally, this increases pt-pmp performance around seven times for dumper eu and 65 times for dumper pteu with no information loss. You need to install elfutils to explore these options.

We plan to publish a blog on eu-stack support in pt-pmp and a new tool, pt-eustack-resolver, soon.

pt-k8s-debug-collector

Users of Percona Operator for PostgreSQL do not need to have psql client installed to collect summaries anymore. After the fix for PT-2233, pt-k8s-debug-collector now uses the client, which is available in the cluster node. Before, it used psql from the client machine.

After the fix for PT-2299, pt-k8s-debug-collector stores details about SSL certificate metadata for all operators. This helps troubleshoot SSL connection issues, such as if one of the certificates is out of date.

pt-stalk

A few years ago, pt-stalk could profile mysqld processes with OProfile‘s opcontrol. This tool was then deprecated and removed from the most currently supported operating systems, which meant that pt-stalk lost a helpful feature. In version 3.6.0, we accepted a pull request from Fernando Ipar that adds similar support but uses a fully supported performance profiler operf.

pt-summary

pt-summary already collects dmidecode information and prints memory and system information out of it. A contribution by Alexey Kopytov adds CPU cache configuration information, based on the dmidecode data, to the tool’s output.

pt-mongodb-summary

pt-mongodb-summary now supports MongoDB 6.0 and newer.

pt-galera-log-explainer

One of the most significant additions to the pt-galera-log-explainer is custom regular expressions. You can now specify them via option –custom-regexes. You also have two options for processing matches; printing a matched string or displaying a custom message.

For example, the following command on the test file node1.20230315.log will print matched strings:

This snippet prints any custom text you want:

The option –skip-merge tells the tool not to merge log files together. This is useful when analyzing logs from different nodes with the same name.

For the PXC operator nodes, instead, the pt-galera-log-explainer introduced a merging mechanism that allows merging log files collected by the pt-k8s-debug-collector multiple times in the same cluster but taken when pvc is destroyed. These logs were originally treated as logs from different nodes, but now they are merged using podname+deployment+namespace as an identificator.

pt-galera-log-explainer now automatically detects Percona Operator for PXC, so the option –pxc-operator is not required.

A new command, whois, prints information about nodes using any type of information, such as nodename, ip, or uuid. By default, the tool autodetects the correct type. If it fails to detect correctly, specify type with option –type.

Here is the sample output:

Community contributions:

This release includes contributions from Community and Percona Engineers who do not actively work on the project. We want to thank:

 

• Alexey Stroganov for eu-stack support in pt-pmp.
• Satoshi MITANI for fixing the wide-character error in pt-online-schema-change.
• Fernando Ipar for operf support in pt-stalk.
• Alexey Kopytov for CPU cache processing in pt-summary.
• Jobin Augustine (Percona Support) for removing the requirement to have a psql client on the node that runs pt-k8s-debug-collector.
• Marco Tusa (Percona Technical Leads) for adding CREATE USER command in pt-show-grants when dumping grants for MariaDB servers.
• Yoann La Cancellera (Percona Support) for all improvements and bug fixes to pt-galera-log-explainer.