Upgrading PostgreSQL – Patroni Cluster and Switching from Python 2 to Python 3

PostgreSQLPython 2 has officially completed its life as of Jan 1st, 2020 and the Python 2.x branch will not be maintained anymore. You might be seeing repeated notices of deprecation in log files and terminals like:

Python 2 had a wonderful life, coming from a humble background and becoming one of the most popular languages and the most popular scripting language. It’s no wonder Patroni-like tools are developed in Python.

Thanks to the growing popularity of Patroni, there are a large number of Patroni cluster setups across different companies. Now “How to migrate from Python 2 to Python 3 with the least possible downtime” becomes a hanging question for Operations teams and DBAs. Additionally, chances are that the cluster will be running an older version of Patroni. If so, the upgrade plan can take up both these challenges at the same time:

  1. Upgrade Patroni Software to Latest release
  2. Switch to Python 3 from Python 2

This blog post is to demonstrate how easy it is, and Patroni’s High Availablity features will be leveraged when we upgrade individual nodes. There are a few key points to note.

  • We should not be disturbing the default Python 2 installation (its binaries and modules) because there could be other programs using them.
  • Python 2 and Python 3 installations will go to different directories so they can coexist in almost all Linux distros.
  • We should avoid multiple Patroni installations (different versions) in a node because they will cause confusion in the long run. So we must remove existing ones.
  • Patroni in each node of the cluster can be using different versions of Python because Patroni is not interfering with PostgreSQL replication.
  • Since Python precompiles scripts it loads to memory for execution, we can replace the Python script while the corresponding program is running in a loop.

Verification of the Version

We shall verify the version of Python and Patroni we use before proceeding. For example:

As we see in this node, Patroni invoked Python from /bin/python. Commonly, this will be a symbolic link to the actual Python binary of a specific version.

We won’t be replacing this because it may break other Python-based programs in the system. It’s worth noting the exact Python version.

Upgrade

Since Python 3 binaries go to a different location in the system, there is nothing stopping us from installing it. It is not going to affect Patroni anyway. I would suggest using the Python from the repository of the Linux distribution.

On CentOS/RHEL, you can use the yum package manager for installing Python binaries and development packages and C compiler as follows:

The above-mentioned step may change depending on the Linux distribution. For example, on Ubuntu, you may install

and for the remaining installation, we can use pip for Python 3 (pip3) and execute the following commands one by one:

These steps may remain the same for all environments.

IMPORTANT: In this blog post, steps are demonstrated with a fresh, minimal installation of Python 3. Mixing pip installation with the package manager of Linux distribution can be dangerous. Please follow either of the ways to install python packages/modules. It is recommended to avoid installing additional Python packages in a Patroni node as the root user.

Now we are entering the tricky part: uninstalling and installing Patroni while it is running. This is possible because Python does a precompilation of Python code and loads it to memory for execution. So replacing the Python source is almost equivalent to replacing a source file of a compiled program.

The Patroni in the system might have installed in different ways. RPM/DEB packages, Patroni Source code from Git repo of the project, or pip installation. Use the corresponding uninstall method for the same. In case your installation is using RPM/DEB packages, the package managers like YUM or APT provides their own removal options

Note: Percona Distribution for PostgreSQL provides RPM and DEB packages for installation of Patroni.

Even if the existing installation of Patroni is installed using easy_install or source install, pip can help you to remove it.

As we can see in this case, we uninstalled an older version 1.5.3. It’s good practice to make sure that Patroni no longer exists in the previous location:

Now we can install the new version of Patroni with Python 3. As of writing this blog, the latest release is 1.6.3.

The new location of Patroni can be different, but make sure to verify the Python version:

During this entire operation of uninstalling and reinstallation of the new version of Patroni, old Patroni will be running. No shutdown is required until this stage. But now the time has come and it’s a matter of new Patroni taking over the PostgreSQL.
Make sure that new Patroni is in the path and service files are updated accordingly.

Now we should be stopping the Patroni service and starting it with new Patroni. Most administrators prefer to manage the Patroni service via Systemd.  In case your environment is also managed by systemd, the following steps can be used. As the first step, verify the systemd service configuration (unit file). If Patroni is located in a different directory, please do the required changes to the path.

You can open the systemd service in your favorite text editor:

However, the best way to edit the service configuration file is to use systemctl’s edit option like

which will open the service configuration for you for editing.

There will be a line mentioning the command to be executed to start the service (ExecStart). Edit this as per the Patroni location.

Once editing is completed, reload the configuration:

Now we shall restart the Patroni deamon. This will cause the associated PostgreSQL instance to go down and start up again.

Finally, we shall verify that the node joined back to the cluster with new Patroni.

We may explicitly verify that the Patroni is started with Python 3 and we have the latest version:

This blog post demonstrates how the upgrade of Python and Patroni can be achieved with the least disruption to service. Only Patroni in one node of the cluster needs to be restarted at a time. We shall repeat the above-mentioned steps in other nodes, also one by one, to upgrade the entire cluster.

Share this post

Comments (3)

  • Kostiantyn Nemchenko (@_knemchenko) Reply

    Have you tried to put Patroni into a pause mode ?https://patroni.readthedocs.io/en/latest/pause.html

    It will allow to shut down patroni daemon on all cluster members without stopping the Postgres instance it is managing.

    January 17, 2020 at 3:29 pm
  • Lyrics Reply

    Hey Jobin, I have some irresistible questions. Is Python a software like wordpress which needs to be updated regularly or is a programming language? This question came to my mind because you said earlier “Python 2 has officially completed its life as of Jan 1st, 2020 and the Python 2.x branch will not be maintained anymore.” What the Python really is that needs to be maintained and who maintains it.
    Sorry for the dumb questions.

    January 18, 2020 at 12:36 am
  • Jobin Augustine Reply

    Thank you Kostiantyn, for that valuable comment. very much appreciate that.
    It will be very useful for everyone who is using Patroni. I shall try that option and write about.
    Thank you once again.
    Jobin.

    January 18, 2020 at 6:27 am

Leave a Reply