EmergencyEMERGENCY? Get 24/7 Help Now!

Containing your logical backups: mydumper in docker

 | September 21, 2015 |  Posted In: Docker, MySQL, Percona Backup Service, Percona XtraBackup

PREVIOUS POST
NEXT POST

Even with software like Percona Xtrabackup, logical backups remain an important component of a thorough backup strategy. To gather a logical backup in a timely fashion we rely on a tool called mydumper. In the Percona Managed Services department we’re lucky enough to have one of the project’s most active contributors and many of the latest features and bug fixes have been released to satisfy some of our own use cases. Compiling mydumper remains one of my personal bug bears and undoubtedly the highest barrier to entry for many potential adopters. There are a few conditions that make compiling it difficult, tiring or even prohibitive. The open source logical backup tool does not supply any official packages, however our friends over at TwinDB are currently shipping a package for CentOS/RHEL. So what if you’re running something else, like Debian, Ubuntu, Arch? Well recently I had a flash of inspiration.

Since I’ve been turning some ideas into docker containers, it dawned on me that it would be a trivial image to create and would add instant portability to the binaries. With a docker environment you can take an image and run a logical backup through to a mapped volume. It’s almost as easy as that.

mydumper_docker

So let me show you what I mean. I have built a docker image with the mydumper, myloader and mysql client libraries installed and it’s available for you on docker hub. This means that we can call a new container to make our backup without technically installing mydumper anywhere. This can get you from zero to mydumper very fast if there are hurdles in your way to deploying the open source backup tool into production.

With the grand assumption that you’ve got Docker installed somewhere, lets pull the image from the docker hub

Once all the layers download (it’s < 200MB) you’re all set to launch a backup using the mydumper binary. You can roll your own command but it could look similar to;

If you’re unfamiliar with Docker itself; a very high level summary for you; Docker is a product intended to facilitate process isolation or ‘micro services’ known as containerization. It intends to be lighter and more efficient than Virtual Machines as we traditionally know them. There’s much more to this work flow than I intend to explain here but please see the further learning section in the footer.

Let me explain a little of the above call. We want to launch a mydumper run isolated to a container. We are giving the docker daemon the instruction to remove the container after it finishes it’s run (–rm), we are calling the container to be an ‘instance’ of the mysqlboy/mydumper image and we are passing a traditional mydumper command as the container’s instruction. We have mapped a location on the local filesystem into the container to ensure that the backup persists after the container is stopped and removed. The mydumper command itself will make a full backup of the instance you point it to (mydumper can make remote backups, pulling the data locally) and will use the less locking, savepoints and compression features.

What’s more, the beauty of containerizing the mydumper/myloader binaries mean that you can use this image in conjunction with docker machine to source logical backups from Mac and Windows where this process is typically difficult to assume.

I’m going to be filling in for my colleague Max Bubenick this week at Percona Live in Amsterdam talking about Logical Backups using Mydumper and if you’re planning to attend, the mydumper docker image will provide you with a quick path to trial. Thanks for reading and if you’re in Amsterdam this week don’t forget to say hello!

Further learning about docker:

https://www.docker.com/whatisdocker

https://www.youtube.com/user/dockerrun

http://europe-2015.dockercon.com/

PREVIOUS POST
NEXT POST
Andrew Moore

Since fall 2013, Andrew has been working within Percona's Remote DBA team plying his experience to the client's environments and internal tools developed to keep operations slick. He lives in the UK with his young family and loves to complain about the less than perfect climate. Andrew makes time to pursue an amateur soccer career but won't be trading in MySQL any time soon.

4 Comments

  • Sounds like a good tool for logical backups. Thanks for sharing the process here! Losing data is annoying at best and disastrous at worst, but there are things you can do to prevent or recover it.

  • Luckily the guys at openSUSE are on the job, so you can grab mydumper for openSUSE/SLES at: https://software.opensuse.org/package/mydumper?search_term=mydumper

    (“Unstable” but that really means that it’s packaged by a contributor)

  • @Miuku: That’s great news. What’s the collective term for openSUSE users? SUSians? Well that’s great they have a simple path for use and I hope that they pick up the new release once Max has it ready, there are some great features coming.

  • I’ve pushed a new image to docker hub. Please find a revised container image with the latest 0.9.2 version of mydumper ready for use @ https://hub.docker.com/r/mysqlboy/docker-mydumper/

Leave a Reply