Buy Percona ServicesBuy Now!

Data Recovery Policy

Introduction

Percona has a long history of assisting customers with data loss incidents involving their database servers. Unfortunately, it's not always possible to successfully recover data. In many cases, even attempting the process takes quite a bit of time. These types of problems are often most appropriately handled as a custom-scoped project initiated through Consulting due to the time involved. However, Percona Support does include assistance with using open source data recovery tools or with restoring a backup taken using Percona XtraBackup. This document is an attempt to clarify what we can do and what we can't do.

Summary of Our Data Recovery Abilities

  • We cannot provide data recovery guarantees, as data recovery is tricky even if you have a backup.
  • If the customer has a working backup of their data, we can assist them through Percona Support:
    • This requires an active support subscription
    • This is an S1 case, 15-30 minute response time depending on support tier
  • If the customer does not have a working backup of their data, and they have experienced database corruption reported during MySQL startup or error logs:
    • This is a complex case and we cannot guarantee recovery of all the customer's data
    • This will need to be billed in full-day increments as a custom-scoped consulting engagement, and we won't be able to begin immediately as we need to schedule the engagement
    • These cannot be performed as T&M, they require a scoping call prior to engagement to ensure we correctly understand the requirements – it may be more appropriate to refer to a third-party instead (such as TwinDB)
    • These cases usually take 2-3 days to resolve, and require the time of a senior consultant, so the customer should expect to be billed for this service (usually this runs $5k or more)
  • If the customer does not have a working backup of their data and they've experienced any sort of physical equipment failure (such as their disks, SAN, RAID array, etc.):
    • We are unable to assist in this case as this requires extremely specialized licensing, equipment, facilities and expertise
      • There are several reputable vendors who have clean room environments and the specialized knowledge and tools to help
      • We don't have any particular partnerships, but can give the customer a list of names of those they can contact (see below)
      • The customer should first contact their storage equipment vendor to see if the vendor has recovery services available or can recommend a specific third-party vendor

Data Recovery Under Percona Support (Emergency Data Recovery)

As part of supporting open source data recovery tools (including Percona XtraBackup) our support teams are able to assist customers with performing "emergency data recovery" if they have a current support subscription with Percona. There are some strict limitations on what can be done (without a deep dive into the specific environment):

  • We can assist the customer with advice on configuring and using open source data recovery tools
  • We can assist the customer with understanding Percona XtraBackup, MyDumper, and mysqldump
  • If the customer has a recent logical backup (created using MyDumper or mysqldump) or physical backup (created with XtraBackup) we can assist them with restoring their data from this backup
  • If the customer is using Amazon EC2 or Amazon RDS, we can provide them advice on creating or restoring a snapshot using tools like ec2-consistent-snapshot and xfs_freeze or interacting with RDS snapshots using aws-cli
  • We can assist the customer with advice on using things like innodb_force_recovery or tools such as undrop-for-innodb at the Support tech's discretion (these items typically should be referred to consulting)
  • We cannot  guarantee that customer data will be restored or recovered – due to the nature of disaster recovery and backups, there are no guarantees

Data Recovery Under Percona Consulting (Advanced Data Recovery)

As part of a custom-scoped consulting engagement, we can go much further (including sending a consultant on-site if necessary). During that time we will provide "advanced data recovery", which includes the consultant attempting a hands-on recovery process and using any tools of which they have knowledge. There are some limitations:

  • We can provide hands-on action to configure and use open source data recovery tools
  • We can provide hands-on action to set up and use Percona XtraBackup, MyDumper, and mysqldump
  • If the customer has a recent logical backup (created using MyDumper or mysqldump) or physical backup (created with XtraBackup) we can provide hands-on action to restore their data from this backup
  • If the customer is using Amazon EC2 or Amazon RDS, we can provide hands on assistance in creating or restoring snapshots, including helping them with automation using tools like ec2-consistent-snapshot or interacting with RDS snapshots using aws-cli
  • In the case that a customer has a corrupted database using a supported storage engine and has no backup available, we will provide hands-on assistance in recovering that database to the best of our abilities:
    • This is using settings like innodb_force_recovery and table repairs (which can be destructive if not understood) with the expertise Percona brings to try to repair a corrupt database to get MySQL to start
    • This also includes tools like undrop-for-innodb from TwinDB which we can use to sometimes recover a dropped database if it was recent.
    • This doesn't cover corruption caused by hardware failures, as these types of issues require specialized equipment and skills Percona cannot provide
  • We cannot  guarantee that customer data will be restored or recovered – there is no guarantee and this is often a multi-day engagement (usually incurring $5k+ in consulting fees)
  • This does NOT include "forensic data recovery"
  • These services are at best a 50/50 chance if the customer didn't have a working backup – there aren't any guarantees
  • As these services require a Consulting engagement, they will occur when a consultant can be scheduled. (we cannot expedite this process)

Data Recovery Services Percona Does NOT Offer (Forensic Data Recovery)

For customers that need assistance that goes beyond working inside of the database ecosystem to perform data recovery, there are techniques available that require specialized equipment, specialized expertise and in many cases government-mandated licensing to perform. These services collectively are referred to as "Forensic Data Recovery" and include using commercial specialized equipment and tools, performing disk scanning techniques and physical reconstruction of data using an Electronic Scanning Microscope (ESM). These are far outside the capabilities of Percona and we do not have any partnerships with vendors of these services to specifically recommend. These are very involved and specialize services:

  • We do not provide any physical data recovery services
  • We are not licensed in any jurisdiction to perform "computer forensics"
  • We can provide a list of reputable data recovery companies to the customer, but we cannot recommend with any prejudice a specific third-party vendor
  • We cannot  guarantee that data will be restored or recovered in these circumstances, regardless of what actions the customer takes with Percona or with a third-party vendor

Conclusion

Percona wants to assist our customers as much as possible especially in trying times, but there are specific and strict limitations to what we can do to help.

In a data loss incident, the best thing is having working recent backups. In this case, the customer can get assistance from Percona Support to perform an "emergency data recovery". If the customer doesn't have a working recent backup, then it requires a custom consulting engagement scheduled when the consultant is available and will cost additional consulting fees and consultant time to attempt a recovery without using a backup. We cannot provide any guarantee of success regardless of the method used. We cannot recommend any third-party vendors as preferred for performing physical forensic data recovery. Third-party services may be required if Percona is unable to recover the data via a consulting engagement. Data loss incidents are difficult and data recovery is a specialized field that goes far beyond expertise in database software. While Percona will try to assist customers who have experienced data loss, we cannot guarantee a successful result.

Visit Percona Store