The other day on a call, a client asked me an interesting question. We were discussing some testing they were doing in Amazon Relational Database Service (RDS). The question came up “since RDS is managed, how can I prove to my security team that the data is actually encrypted?” We’ve all read whitepapers and blogs that talk about encryption in the cloud, but it is something I’ve not needed to validate before. Instead, I just enable encryption when I create an RDS instance and move along to the next steps of the setup. This discussion really got me thinking – what is the best way to enable and test encryption at rest in my RDS instance?
Encryption at Rest – MySQL Fundamentals
Before getting too far into the RDS specifics, I wanted to cover the basics of encryption at rest in MySQL. There are essentially two ways to encrypt data at rest:
- Full disk encryption (filesystem/block level)
- Transparent Data Encryption (TDE) with InnoDB
Full disk encryption is just like it sounds – the entire disk (or data directory mount point) is encrypted and a key is needed to read the data. Without this key, MySQL cannot read the data files and is rendered useless. This is a very straightforward way to maintain a flexible encryption setup as you can switch out storage engines or database servers altogether.
Transparent Data Encryption (TDE) was introduced in MySQL 5.7 and handles encryption at the instance level as opposed to the OS level. By enabling a master key (using the keyring_file plugin or a remote service such as Vault), the InnoDB engine will encrypt its own data to disk. Without access to the key, InnoDB will fail to load within MySQL. Likewise, any file-level backups (i.e. using Percona XtraBackup) will be rendered useless without the key.
Both methods are completely acceptable forms of encryption at rest, but different organizations may choose one or the other based on their specific requirements. When looking at RDS, however, things are different.
Encryption at Rest – RDS
As RDS is a managed service, you do not interact with the base operating system or filesystem. Rather, you leverage the API provided by AWS to launch and configure your instance. This poses a problem if you are currently using a keyring_* plugin in MySQL – how do you store your master key on the filesystem or configure other plugins for remote key management? Unfortunately, you can’t. However, there is still a viable solution for encrypting your data at rest with RDS.
With RDS being built on the underlying IaaS components of AWS, you are able to leverage the existing functionality of those components through the API. The feature we are looking at here is EBS (Elastic Block Store) encryption. Much like the full disk encryption method described above, EBS allows you to encrypt volumes using AWS default keys or customer master keys (CMKs) defined within the Key Management Service (KMS). These keys are required for the volume to be usable and for snapshots to be restorable. This enables you to ensure data-at-rest encryption for your RDS instances.
Putting it All Together
In the past, I’ve used both the RDS default encryption key and user-defined keys. For this exercise, however, I wanted to ensure I could prevent my database from being restored or restarted on demand. To start, I created a new CMK in the KMS service to use with my database and gave the RDS service permission to access it:
Then, when I create my RDS instance, I can choose this new key when I enable encryption. For my test, I encrypted my instance using a cleverly named CMK key called database-key:
Note that along with my CMK, the (default) aws/rds key is an option. I want control over my key and when it is used so I choose my key and not the default. Note that certain instance types don’t offer encryption so if you don’t see this option when creating your instance, check your instance type against the list here.
After that, it is just business as usual. I connected to my RDS instance, populated some dummy data, and took my first snapshot. This is when it was time to confirm that I could indeed disable the key and prevent access to my data. Here is the important thing to note – as my instance was encrypted with a specific key at startup, the snapshots also require the same key. You can see that in my Snapshot Restore window, it specifies the key needed and I don’t have the option to disable it or choose another key:
This is how RDS ensures that the key is tied to the instance and you can’t just take the data and restore it elsewhere. After disabling my CMK (with my instance still running), I tried to restore my snapshot. Success! My snapshot was not able to be restored and I was pleasantly greeted with the following error message:
This confirms that by using my own key, I can effectively lock out any unauthorized restoration or access to the data at the disk level. But now, I wanted to verify one additional item. Looking at how encryption at rest works on a standard instance, I also wanted to confirm that the key was needed to start the instance.
With the key disabled, I stopped the running instance. After that process completed, I tried to restart my instance. Again, it was successful in that I was not able to restart my instance with the key disabled:
So just like a standard instance that is missing the master key, the RDS instance isn’t able to start with the key disabled. Similar to an on-prem deployment, ensuring that your CMK is secure is outside of the scope of this blog post. But with that being said, this exercise demonstrates the power and flexibility of using CMKs to ensure your data is encrypted at rest.
Hopefully, this post and example were useful to you. It is always a good practice to ensure that documentation matches behavior. In this case, I feel confident that leveraging CMKs to control my RDS encryption should meet a majority of security requirements. For any help ensuring that your RDS deployment is following best practices, please don’t hesitate to reach out to Percona. We’d be happy to help ensure your RDS (or other cloud) deployment is done in the best possible way!
For more information, download our solution brief below which outlines setting up MySQL Amazon RDS instances to meet your company’s growing needs. Amazon RDS is suitable for production workloads and also can accommodate rapid deployment and application development due to the ease of initial setup.