The Steps Involved in Creating a Percona Product Release

Have you ever wondered what it takes to complete Percona Server for MySQL (PS), Percona XtraDB Cluster (PXC), and Percona XtraBackup (PXB) releases? 

Let’s step back just a minute and talk about what Percona stands for. We believe we “stand on the shoulders of giants.” This means we respect our upstream sources and work to add value to the base products. Over time, new functionality is added. Much of this value-add was implemented on the 5.7 series and pulled forward to the 8.0 series. Each time we receive an upstream release, we must reapply these features to the release we receive. This process is what we call the Merge Process. There are close to 40 add-on features being maintained with each release. 

Creating a Percona Product Release

The Merge Process

The Merge Process is completed first for Percona Server for MySQL (PS), and then the merged product is used as the basis for merging the release for Percona XtraDB Cluster (PXC).  In addition, Percona XtraBackup (PXB) is based on Oracle’s MySQL. 

The merge process actually applies the Oracle MySQL commits for a received release to the base Percona Server for MySQL (PS) code. Then the validation process is begun.

For each Oracle commit, an Engineer reviews the code, what it will do, and verifies there will be no adverse effect to the base PS code. In some cases, the Oracle code will replace the existing Percona Server for MySQL (PS) code (Percona may have added the same functionality previously or fixed a bug) or could cause conflicts with the existing Percona code. Each situation is then evaluated and resolved. For the last three 8.0 releases, there have been 2000-2500 commits and 50-100 for the 5.7 base in each Oracle release.

Validation and Testing

Once all commits have been applied to the Percona Server for MySQL (PS) previous base, the validation/testing process begins. This process uses automated testing to validate the new codebase. We execute 1500-2000 regression-type test cases in addition to specifically validating bug fixes and new functionality. This process can take anywhere from 5-10 days for a 5.7 merge and between 35-40 days for an 8.0 merge.

Once the Percona Server for MySQL (PS) merge is completed, the new base is used for the Percona XtraDBCluster (PXC) merge, which combines the new Percona Server for MySQL (PS) base with any Percona XtraDBCluster (PXC) specific functionality, using the same process as Percona Server for MySQL (PS). In addition, if a Codership (Galera) release is also available, we go through a slightly different process. In the case of Codership, beginning with the 8.0.19 release, there are no more git commits (just a source tarball), and we have to use release notes to point to the places we need to look for changes. The 5.7 codebase uses Galera 3.0 (still use committed work), and the 8.0 codebase uses the 4.x base (currently on 4.6, soon to go to 4.7). This also includes the same level of validation. This process again can take anywhere from 5-10 days for a 5.7 merge and between 35-40 days for an 8.0 merge.

The last of the merge processes is to evaluate and complete processing for Percona XtraBackup (PXB). The first thing we do when Oracle releases a new wave of its product updates is verify whether the most recent Percona XtraBackup (PXB) – the one from the previous wave – is compatible with the new MySQL Server release. If we find compatibility issues, they will be documented and resolved, and be included in the merge process. The Percona XtraBackup (PXB) merge is done in the same way as Percona Server for MySQL (PS) for a small portion of the Oracle MySQL codebase. Percona XtraBackup (PXB) uses functionality related to redo log processing, such as writes/read/parsing and checksum processing for the read/write process. This process is done concurrently with the Percona Server for MySQL (PS) process and is done for both the 2.4 and 8.0 versions of Percona XtraBackup (PXB).  

For all three products, security (CVE) modifications are purposefully not exposed individually. They are included and noted, but you will not find specific commits for them.

We understand that choosing open source software for your business can be a potential minefield. You need to select the best available options, which fully support and adapt to your changing needs. In this white paper, we discuss the key features that make open source software attractive, and why Percona’s software might be the best option for your business.

Download “When is Percona Software the Right Choice?”

Share this post

Comments (2)

  • Francisco Miguel Biete Banon Reply

    Why are PXC 8.0 merges so much longer than 5.7? Codership also provides a patch for 8.0, and PS 8.0 doesn’t have so many Percona changes as 5.7 does… It should take less effort fo apply the galera patch to 8.0 than 5.7

    About CVE not being committed individually, I have never liked this approach. Security by obscurity. And anyway it’s extremely trivial to identify the CVE fix commit, at least when it impacts PXC and not PS.

    March 29, 2021 at 5:54 am
  • Kathy Williamson Reply

    Hi Francisco – I wanted to get back to you with some clarifications.

    Codership only provides a source tarball as of the 8.0.19 release for the 4.x series. This results in a great deal of analysis work to determine what has changed and how it must be integrated. The 3.x series remains as it was with specific commits provided.

    PS 8.0 pulled forward over 40 of the Percona-specific changes from 5.7. Each of these must be evaluated and verified with each 8.0 release

    Oracle releases 50-100 commits for 5.7 and 2000-2500 for 8.0. The amount of change is very different and no new functionality is released in 5.7. As a result the analysis time required is much greater.

    So you see there is a great deal more time and effort involved in the 8.0 process compared to the 5.7 merge process.

    Thanks for you feedback

    March 30, 2021 at 10:44 am

Leave a Reply