Percona XtraDB Cluster 5.6 has been GA for several months now and people are thinking more and more about moving from 5.5 to 5.6. Most people don’t want to upgrade all at once, but would prefer a rolling upgrade to avoid downtime and ensure 5.6 is behaving in a stable fashion before putting all of production on it. The official guide to a rolling upgrade can be found in the PXC 5.6 manual. This blog post will attempt to summarize the basic process.
However, there are a few caveats to trying to do a rolling 5.6 upgrade from 5.5:
The basic upgrade flow is:
This is, in essence, exactly like upgrading a 5.5 master/slave cluster to 5.6 — you upgrade the slaves first, promote a slave and upgrade the master; we just have more masters to think about.
Once your upgrade is fully to 5.6, then you can go back through and remove all the 5.5 backwards compatibility
The heaviest caveat is probably the fact that in a mixed 5.5 / 5.6 cluster, you are not supposed to write to the 5.6 nodes. Why is that? Well, the reason goes back to MySQL itself. PXC/Galera uses standard RBR binlog events from MySQL for replication. Replication between major MySQL versions is only ever officially supported:
This compatibility requirement (which has existed for a very long time in MySQL) works great when you have a single Master replication topology, but true multi-master (multi-writer) has obviously never been considered.
The restriction on 5.6 masters of 5.5 slaves is probably too strict in many cases. Technically only older to newer replication is ever truly supported, but in practice you may be able to run a mixed cluster with writes to all nodes as long as you are careful. This means (at least) that any modifications to column type formats in the newer version NOT be upgraded while the old version remains active in the cluster. There might be other issues, I’m not sure, I cannot say I’ve tested every possible circumstance.
So, can I truly say I recommend this? I cannot say that officially, but you may find it works fine. As long as you acknowledge that something unforeseen may break your cluster and your migration plan, it may be reasonable. If you decide to explore this option, please test this thoroughly and be willing to accept the consequences of it not working before trying it in production!
Another alternative is rather than trying to mix the clusters and keeping 5.6 nodes read_only, why not just setup the 5.6 cluster as an async slave of your 5.5 cluster and migrate your application to the new cluster when you are ready? This is practically the same as maintaining a split 5.5/5.6 read_write/read_only cluster without so much risk and a smaller list of don’ts. Cutover in this case would be effectively like promoting a 5.6 slave to master, except you would promote the 5.6 cluster.
One caveat with this approach might be dealing with replication throughput: async may not be able to keep up replicating your 5.5 cluster writes to a separate 5.6 cluster. Definitely check out wsrep_preordered to speed things up, it may help. But realize some busy workloads just may not ever be able to use async into another cluster.
A final alternative for this post is the idea of simply upgrading the entire cluster to 5.6 all at once during a maintenance window. I grant that this defeats the point of a rolling upgrade, but it may offer a lot of simplicity in the longer run.
A rolling PXC / Galera upgrade across major MySQL versions is limited by the fact that there is no official support or reason for Oracle to support newer master to older slave. In practice, it may work much of the time, but these situations should be considered carefully and the risk weighed against all other options.
Percona’s widely read Percona Data Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.
Besides specific database help, the blog also provides notices on upcoming events and webinars.
Want to get weekly updates listing the latest blog posts? Subscribe to our blog now! Submit your email address below and we’ll send you an update every Friday at 1pm ET.