New mydumper 0.6.1 release offers performance and usability features

One of the tasks within Percona Remote DBA is to ensure we have reliable backups with minimal impact. To accomplish this, one of the tools our team uses is called mydumper. We use mydumper for logical backups because of several nice features. Some of them are:

  • multithreaded, producing very fast backups compared to mysqldump
  • almost no locking, if not using non innodb tables
  • built-in compression
  • separate files for each table, making it easy to restore single tables or schema, and the possibility to hardlink files reducing the space needed to keep history of backups. Also this feature give you the possibility to restore with more than one thread.

The mydumper project was started at the beginning of 2009 by Domas Mituzas, and a few months ago we started collaborating and adding new features to improve performance and usability.

And now we are glad to announce our second release: mydumper 0.6.1. You can download the source code here. It is highly recommended to upgrade it if you are using 0.6.0 as we fixed and improved the new less-locking feature.

New features in the mydumper 0.6 series:

  • Consistent backups with less locking
    This new feature consists of locking all non-innodb tables with the dumping threads at the beginning so in this way we can unlock the flush tables with read lock earlier and no need to wait until all non-innodb tables were dumped. You can take advantage of this feature when you have large archive or myisam tables.
  • File size chunks
    Now you can split tables dump into different files with fixed size. This is usefull to reduce storage capacity needed to keep history backups by using hardlinks. Think on big “log” tables or tables where old data didnt change, now you will be able to hardlink back those chunks.
  • Metadata Locking
    Added new option –use-savepoints to reduce metadata locking issues while backup is running.

Fixed Bugs in the mydumper 0.6 series:

mydumper 0.6.0

  • #1250269 ensure statement is not bigger than statement_size
  • #827328 myloader to set UNIQUE_CHECKS = 0 when importing
  • #993714 Reducing the time spent with a global read lock
  • #1250271 make it more obvious when mydumper is not successful
  • #1250274 table doesnt exist should not be an error
  • #987344 Primary key name not quoted in showed_nulls test
  • #1075611 error when restoring tables with dashes in name
  • #1124106 Mydumper/myloader does not care for 0-s in AUTO_INCREMENT fields
  • #1125997 Timestamp data not portable between servers on differnt timezones

mydumper 0.6.1

  • #1273441 less-locking breaks consistent snapshot
  • #1267501 mydumper erroneously always attempts a dummy read
  • #1272310 main_connection keep an useless transaction opened and permit infinite metadata table lock
  • #1269376 mydumper 0.6.0 fails to compile “cast from pointer to integer of different size”
  • #1265155 create_main_connection use detected_server before setting it
  • #1267483 Build with MariaDB 10.x
  • #1272443 The dumping threads will hold metadata locks progressively while are dumping data.

Note: #1267501 fix is important for any galera cluster deployment, because of this bug #1265656 that was fixed and released in Percona XtraDB Cluster 5.6.15-25.3.

Share this post

Comments (16)

  • Roman Vynar

    Great job!

    March 10, 2014 at 1:15 pm
  • George

    Unfortunately seems still broken install for CentOS 6.x builds up to 6.5 🙁

    March 10, 2014 at 3:06 pm
  • Daniel

    A very anticipated release. It’s great to see that the mydumper project gets renewed

    March 10, 2014 at 5:01 pm
  • Khalid

    On Ubuntu Server 12.04 LTS, you need to get the dependencies listed here before MyDumper would build successfully without any errors: Fast parallel MySQL backups and imports using MyDumper.

    The patch is no longer needed with 0.6.1, so ignore that part.

    March 10, 2014 at 5:20 pm
  • Max Bubenick

    George this release is focused mainly in new features, we are working on make it easier to compile on latest versions.

    March 10, 2014 at 5:32 pm
  • George

    Thanks Max very much appreciate the effort as it’s good to see mydumper development continuing as it’s definitely a useful tool !

    Also any chance of getting back the domain, seems to missing in action so I uploaded the old docs for reference at right now.

    Looking forward to being able to use mydumper on CentOS 6.x again with MySQL 5.5/5.6, MariaDB 5.5/10 and Percona 5.5/5.6 🙂

    March 10, 2014 at 10:16 pm
  • Rick James

    Does it work on Windows? If so, what caveats exist?

    March 11, 2014 at 7:27 pm
  • Max Bubenick

    Rick, I’ve never tried, cygwin could have all necessary packages but not sure

    March 12, 2014 at 3:37 pm
  • Rick James

    XtraBackup is not maintained on Windows, for some reason. And you mention multiple threads, which gets tricky in Windows.

    March 12, 2014 at 4:11 pm
  • Nils

    Any chance off getting deb/rpm packages for this?

    April 9, 2014 at 5:15 pm
  • Max Bubenick

    Nils, not at this time, but we have plans for it in the near future

    April 11, 2014 at 1:11 pm
  • Kubilay

    Max, how near is the near future for .deb ? 🙂



    April 26, 2014 at 3:28 pm
  • astrostl

    Is there a reason that –less-locking isn’t a default setting? Makes me curious about downsides, if any. Thx!

    June 4, 2014 at 4:44 pm
  • adil

    this is very good, one thing i want to ask,is this backup result can be use for creating slave database?

    May 19, 2015 at 4:54 am
  • Max Bubenick

    Sure, you can get the binlog position from the metadata file

    May 19, 2015 at 8:26 am
  • Lakshma

    Do we still get performance benefit using mydumper over mysqlpump? Did anyone checked?

    November 10, 2015 at 12:32 pm

Comments are closed.

Use Percona's Technical Forum to ask any follow-up questions on this blog topic.