November 1, 2014

New mydumper 0.6.1 release offers performance and usability features

One of the tasks within Percona Remote DBA is to ensure we have reliable backups with minimal impact. To accomplish this, one of the tools our team uses is called mydumper. We use mydumper for logical backups because of several nice features. Some of them are:

  • multithreaded, producing very fast backups compared to mysqldump
  • almost no locking, if not using non innodb tables
  • built-in compression
  • separate files for each table, making it easy to restore single tables or schema, and the possibility to hardlink files reducing the space needed to keep history of backups. Also this feature give you the possibility to restore with more than one thread.

The mydumper project was started at the beginning of 2009 by Domas Mituzas, and a few months ago we started collaborating and adding new features to improve performance and usability.

And now we are glad to announce our second release: mydumper 0.6.1. You can download the source code here. It is highly recommended to upgrade it if you are using 0.6.0 as we fixed and improved the new less-locking feature.

New features in the mydumper 0.6 series:

  • Consistent backups with less locking
    This new feature consists of locking all non-innodb tables with the dumping threads at the beginning so in this way we can unlock the flush tables with read lock earlier and no need to wait until all non-innodb tables were dumped. You can take advantage of this feature when you have large archive or myisam tables.
  • File size chunks
    Now you can split tables dump into different files with fixed size. This is usefull to reduce storage capacity needed to keep history backups by using hardlinks. Think on big “log” tables or tables where old data didnt change, now you will be able to hardlink back those chunks.
  • Metadata Locking
    Added new option –use-savepoints to reduce metadata locking issues while backup is running.

Fixed Bugs in the mydumper 0.6 series:

mydumper 0.6.0

  • #1250269 ensure statement is not bigger than statement_size
  • #827328 myloader to set UNIQUE_CHECKS = 0 when importing
  • #993714 Reducing the time spent with a global read lock
  • #1250271 make it more obvious when mydumper is not successful
  • #1250274 table doesnt exist should not be an error
  • #987344 Primary key name not quoted in showed_nulls test
  • #1075611 error when restoring tables with dashes in name
  • #1124106 Mydumper/myloader does not care for 0-s in AUTO_INCREMENT fields
  • #1125997 Timestamp data not portable between servers on differnt timezones

mydumper 0.6.1

  • #1273441 less-locking breaks consistent snapshot
  • #1267501 mydumper erroneously always attempts a dummy read
  • #1272310 main_connection keep an useless transaction opened and permit infinite metadata table lock
  • #1269376 mydumper 0.6.0 fails to compile “cast from pointer to integer of different size”
  • #1265155 create_main_connection use detected_server before setting it
  • #1267483 Build with MariaDB 10.x
  • #1272443 The dumping threads will hold metadata locks progressively while are dumping data.

Note: #1267501 fix is important for any galera cluster deployment, because of this bug #1265656 that was fixed and released in Percona XtraDB Cluster 5.6.15-25.3.

About Max Bubenick

Max Bubenick has been working with MySQL for more than 10 years. Before joining Percona's Remote DBA team in 2013 he worked as lead DBA for one of the biggest social gaming companies at that time. Also he maintains mydumper.

Comments

  1. Roman Vynar says:

    Great job!

  2. George says:

    Unfortunately seems still broken install for CentOS 6.x builds up to 6.5 https://bugs.launchpad.net/mydumper/+bug/1157113 :(

  3. Daniel says:

    A very anticipated release. It’s great to see that the mydumper project gets renewed

  4. Khalid says:

    On Ubuntu Server 12.04 LTS, you need to get the dependencies listed here before MyDumper would build successfully without any errors: Fast parallel MySQL backups and imports using MyDumper.

    The patch is no longer needed with 0.6.1, so ignore that part.

  5. Max Bubenick says:

    George this release is focused mainly in new features, we are working on make it easier to compile on latest versions.

  6. George says:

    Thanks Max very much appreciate the effort as it’s good to see mydumper development continuing as it’s definitely a useful tool !

    Also any chance of getting back the mydumper.org domain, seems to missing in action so I uploaded the old docs for reference at http://centminmod.com/mydumper.html right now.

    Looking forward to being able to use mydumper on CentOS 6.x again with MySQL 5.5/5.6, MariaDB 5.5/10 and Percona 5.5/5.6 :)

  7. Rick James says:

    Does it work on Windows? If so, what caveats exist?

  8. Max Bubenick says:

    Rick, I’ve never tried, cygwin could have all necessary packages but not sure

  9. Rick James says:

    XtraBackup is not maintained on Windows, for some reason. And you mention multiple threads, which gets tricky in Windows.

  10. Nils says:

    Any chance off getting deb/rpm packages for this?

  11. Max Bubenick says:

    Nils, not at this time, but we have plans for it in the near future

  12. Kubilay says:

    Max, how near is the near future for .deb ? :-)

    Thanks

    Kubilay

  13. astrostl says:

    Is there a reason that –less-locking isn’t a default setting? Makes me curious about downsides, if any. Thx!

Speak Your Mind

*