What’s New in Percona Toolkit 3.5.6

December 22, 2023

Author

Sveta Smirnova

Insight for DBAs

Insight for Developers

Percona Software

Share this Post:

Percona Toolkit 3.5.6 was released on December 21, 2023. This blog post covers the main changes in this release.

New tool: pt-galera-log-explainer

We continue adding more instruments for the Support teams. pt-galera-log-explainer was written by my Percona Support colleague Yoann La Cancellera. pt-galera-log-explainer filters, aggregates, and summarizes multiple Galera library logs together. The purpose of this tool is to help find useful information in Percona XtraDB Cluster/Galera library logs.

pt-galera-log-explainer takes raw log files, usually very verbose as the one available in the regression test suite, and makes short summary out of them:

$ ./bin/pt-galera-log-explainer list --all src/go/pt-galera-log-explainer/tests/logs/merge_rotated_daily/node1.20230315.log<br>identifier                     node1                                               <br>current path                   .../tests/logs/merge_rotated_daily/node1.20230315.log   <br>last known ip                                                                          <br>last known name                node1                                               <br>mysql version                                                                          <br>                                                                                        <br>2023-03-15T20:10:57.784904+02:00   node2 joined                                        <br>2023-03-15T20:10:57.785568+02:00   node3 left                                          <br>2023-03-15T20:10:57.791959+02:00   node3 left                                          <br>2023-03-15T20:10:57.797221+02:00   PRIMARY(n=2)                                        <br>2023-03-15T20:20:12.714291+02:00   node2 joined                                        <br>2023-03-15T20:20:12.714331+02:00   node3 joined                                        <br>2023-03-15T20:20:13.776977+02:00   PRIMARY(n=3)                                        <br>2023-03-15T20:20:14.839684+02:00   local node will resync node3                        <br>2023-03-15T20:20:14.839723+02:00   SYNCED -> DONOR                                     <br>2023-03-15T20:20:15.799020+02:00   IST will be used                                    <br>2023-03-15T20:20:16.850525+02:00   finished sending IST to node3                       <br>2023-03-15T20:20:16.850549+02:00   DESYNCED -> JOINED                                  <br>2023-03-15T20:20:16.865312+02:00   JOINED -> SYNCED

$ ./bin/pt-galera-log-explainer list --all src/go/pt-galera-log-explainer/tests/logs/merge_rotated_daily/node1.20230315.log identifier node1 current path .../tests/logs/merge_rotated_daily/node1.20230315.log last known ip last known name node1 mysql version 2023-03-15T20:10:57.784904+02:00 node2 joined 2023-03-15T20:10:57.785568+02:00 node3 left 2023-03-15T20:10:57.791959+02:00 node3 left 2023-03-15T20:10:57.797221+02:00 PRIMARY(n=2) 2023-03-15T20:20:12.714291+02:00 node2 joined 2023-03-15T20:20:12.714331+02:00 node3 joined 2023-03-15T20:20:13.776977+02:00 PRIMARY(n=3) 2023-03-15T20:20:14.839684+02:00 local node will resync node3 2023-03-15T20:20:14.839723+02:00 SYNCED -> DONOR 2023-03-15T20:20:15.799020+02:00 IST will be used 2023-03-15T20:20:16.850525+02:00 finished sending IST to node3 2023-03-15T20:20:16.850549+02:00 DESYNCED -> JOINED 2023-03-15T20:20:16.865312+02:00 JOINED -> SYNCED

The tool can process logs from multiple nodes and draw a timeline. You can find a usage example with sample output in the user reference manual. You can also filter events to have a more compact output.

pt-galera-log-explainer is an excellent tool for analyzing large log files that produce the Galera library.

Better macOS and ARM support

This release contains a few improvements for platforms and operating systems that Percona does not officially support.

PR-516, contributed by Ivan Kruglov, makes pt-online-schema-changeand other tools respect case-insensitive lookup on Windows and macOS: the default option for these operating systems.

PR-720 simplifies the build process for the Percona Toolkit. Historically, Percona Toolkit was written in Perl and Shell programming languages. Build instructions for all code were simple:

perl Makefile.PL<br>make<br>(optionally) make test<br>make install

1	perl Makefile.PL<br>make<br>(optionally) make test<br>make install

Since the first tool, written in the Go programming language, was introduced, package maintainers had to perform one extra step:

cd src/go<br>make <PLATFORM such as linux-amd64>

1	cd src/go<br>make <PLATFORM such as linux-amd64>

Now, this extra step is optional because the top-level Makefilehas instructions for building Go tools on the current platform.

PR-712 adds the darwin-arm64 platform to the list of platforms for which Go binaries could be built. While we do not officially support ARM and macOS, you can build the tools yourself without extra effort.

To create macOS ARM binaries on the same platform, simply run the following from the top-level directory.

perl Makefile.PL<br>make

1	perl Makefile.PL<br>make

If you want to create binaries on another platform, change the directory to src/go, then run

make darwin-arm64

1	make darwin-arm64

We also have unofficial ARM packages for Percona Toolkit and PMM Dump in our labs at https://github.com/Percona-Lab/percona-on-arm.

Tools improvements

Explain output for slow query in JSON report for pt-query-digest

Earlier, pt-query-digest did not print EXPLAIN output with option --output=json . PR-471, sent by Ayush Goyal, adds this possibility.

Hook before_die for pt-online-schema-change

PR-509, sent by Ilaria Migliozzi, introduces a new hook for pt-online-schema-change . If pt-online-schema-change stops ungracefully, this hook lets you print diagnostic information about failing operations. You can find an example plugin on GitHub.

Option –unstop for pt-archiver

pt-archiver supports the option –stop that terminates running instances by creating a sentinel file. However, there was no option to remove this file and restart pt-archiver . PR-429 by fraff resolves this gap by adding the option --unstop . If pt-archiver is called with this option, it removes the sentinel file and restarts the operation.

Quality improvements

In my last Percona Toolkit release blog, What’s New in Percona Toolkit 3.5.5, I wrote about how we plan to make regression tests part of the release process. This partially happened, and we are testing Percona Toolkit with Percona Server for MySQL 5.7 and 8.0 on all supported platforms now. There are still a few issues with tests, as reported at PT-2295. We are planning to fix them and then add support for other products.

Viktor Szépe continued to contribute to the quality of the Percona Toolkit source code. He introduced EditorConfig for our GitHub repository, fixed typos, and made style improvements for Go code.

In addition to Kushal Haldar’s reports about vulnerabilities in Go, we enabled better automation for our GitHub repository. This release was built with the latest version of Go and fixes all known module vulnerabilities.

Percona Toolkit Docker images

We released an official Docker image for the Percona Toolkit at https://hub.docker.com/r/percona/percona-toolkit. It can be installed using the following command.

docker pull percona/percona-toolkit

1	docker pull percona/percona-toolkit

To call any tool using docker, run the following.

docker run <TOOL NAME> <OPTIONS>

1	docker run <TOOL NAME> <OPTIONS>

For example:

$ docker run --network="host" percona/percona-toolkit pt-online-schema-change <br>> h=127.0.0.1,P=12345,u=msandbox,p=msandbox,D=test,t=t1 --alter='ADD COLUMN f2 INT' <br>> --execute<br>Found 2 slaves:<br>s76 -> 127.0.0.1:12346<br>s76 -> 127.0.0.1:12347<br>Will check slave lag on:<br>s76 -> 127.0.0.1:12346<br>s76 -> 127.0.0.1:12347<br>Operation, tries, wait:<br>analyze_table, 10, 1<br>copy_rows, 10, 0.25<br>create_triggers, 10, 1<br>drop_triggers, 10, 1<br>swap_tables, 10, 1<br>update_foreign_keys, 10, 1<br>Altering `test`.`t1`...<br>Creating new table...<br>Created new table test._t1_new OK.<br>Altering new table...<br>Altered `test`.`_t1_new` OK.<br>2023-12-21T15:29:22 Creating triggers...<br>2023-12-21T15:29:22 Created triggers OK.<br>2023-12-21T15:29:22 Copying approximately 1 rows...<br>2023-12-21T15:29:22 Copied rows OK.<br>2023-12-21T15:29:22 Analyzing new table...<br>2023-12-21T15:29:22 Swapping tables...<br>2023-12-21T15:29:22 Swapped original and new tables OK.<br>2023-12-21T15:29:22 Dropping old table...<br>2023-12-21T15:29:22 Dropped old table `test`.`_t1_old` OK.<br>2023-12-21T15:29:22 Dropping triggers...<br>2023-12-21T15:29:22 Dropped triggers OK.<br>Successfully altered `test`.`t1`

$ docker run --network="host" percona/percona-toolkit pt-online-schema-change > h=127.0.0.1,P=12345,u=msandbox,p=msandbox,D=test,t=t1 --alter='ADD COLUMN f2 INT' > --execute Found 2 slaves: s76 -> 127.0.0.1:12346 s76 -> 127.0.0.1:12347 Will check slave lag on: s76 -> 127.0.0.1:12346 s76 -> 127.0.0.1:12347 Operation, tries, wait: analyze_table, 10, 1 copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `test`.`t1`... Creating new table... Created new table test._t1_new OK. Altering new table... Altered `test`.`_t1_new` OK. 2023-12-21T15:29:22 Creating triggers... 2023-12-21T15:29:22 Created triggers OK. 2023-12-21T15:29:22 Copying approximately 1 rows... 2023-12-21T15:29:22 Copied rows OK. 2023-12-21T15:29:22 Analyzing new table... 2023-12-21T15:29:22 Swapping tables... 2023-12-21T15:29:22 Swapped original and new tables OK. 2023-12-21T15:29:22 Dropping old table... 2023-12-21T15:29:22 Dropped old table `test`.`_t1_old` OK. 2023-12-21T15:29:22 Dropping triggers... 2023-12-21T15:29:22 Dropped triggers OK. Successfully altered `test`.`t1`

Docker images are not only needed for those who prefer to install software via docker but will allow us to easily ship the Percona Toolkit together with other software, such as Percona Monitoring and Management or Percona Kubernetes Operators.