This quick post demonstrates using Percona Server for MySQL in Docker Swarm with some new authentication provisioning practices.
Some small changes to the startup script for the Percona-Server container image allows us to specify a file that contains password values to set as our root user’s secret. “Why do we need this functionality,” I hear you cry? When we use an environment variable, it’s not terribly hard to locate the value to which someone has set as their database root password. Environment variables are not well suited for sensitive data. We preach against leaving our important passwords in easy to reach places. So moving towards something more secure whilst retaining usability is desirable. I’ll detail the current methods, the problems, and finish off with Docker Secrets – which in my opinion, is the right direction to be heading.
Environment Variables
I’ll elaborate on the main reason why we would want to change from the default given behavior. In the documentation for using the MySQL/Percona and MariaDB containers, we are invited to start containers with an environment variable to control what the instance’s root password is set as upon startup. Let’s demonstrate with the latest official Percona-Server image from Percona’s repository of images on the Docker Hub registry:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
moore@chom:~$ docker pull percona/percona-server:latest latest: Pulling from percona/percona-server e12c678537ae: Already exists 65ab4b835640: Pull complete f63269a127d1: Pull complete 757a4fef28b8: Pull complete b0cb547a5105: Pull complete 4214179ba9ea: Pull complete 155dafd2fd9c: Pull complete 848020b1da10: Pull complete 771687fe7e8b: Pull complete Digest: sha256:f3197cac76cccd40c3525891ce16b0e9f6d650ccef76e993ed7a22654dc05b73 Status: Downloaded newer image for percona/percona-server:latest |
Then start a container:
|
1 2 3 4 5 6 7 8 9 |
moore@chom:~$ docker run -d --name percona-server-1 -e MYSQL_ROOT_PASSWORD='secret' percona/percona-server d08f299a872f1408c142b58bc2ce8e59004acfdb26dca93d71f5e9367b4f2a57 moore@chom:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d08f299a872f percona/percona-server "/entrypoint.sh " 32 seconds ago Up 31 seconds 3306/tcp percona-server-1 |
Looks good, eh? Let’s inspect this container a little closer to reveal why this method is flawed:
|
1 2 |
moore@chom:~$ docker inspect --format '{{ index (index .Config.Env) 0}}' percona-server-1 MYSQL_ROOT_PASSWORD=secret |
*facepalm*
We don’t want the root password exposed here, not really. If we wanted to use this method in docker-compose files, we would also be storing passwords inline, which isn’t considered a secure practice.
Environment File
Another approach is to use an environment file. This is simply a file that we can provide docker run or docker-compose in order to instantiate the environment variables within the container. It’s a construct for convenience. So just to illustrate that we have the same problem, the next example uses the mechanism of an environment file for our database container:
|
1 2 3 4 5 6 7 8 |
moore@chom:~$ echo 'MYSQL_ROOT_PASSWORD=secret' > /tmp/ps.env moore@chom:~$ docker run -d --name percona-server-2 <em>--env-file</em>=/tmp/ps.env percona/percona-server d5105d044673bd5912e0e29c2f56fa37c5f174d9d2a4811ceaba284092837c84 moore@chom:~$ docker inspect --format '{{ index (index .Config.Env) 0}}' percona-server-2 MYSQL_ROOT_PASSWORD=secret NOTE: shortly after starting this container failed because we didn't provide mysql root password options |
While we’re not specifying it in our docker run command or our docker-compose.yml file, the password value remains on our filesystem within the environment file. Again, not ideal.
Password File
With the ability to use a password file it obscures this from the inspect output. Let’s roll through the steps we would use to leverage this new option. With our new Percona-Server image, we’re going to start a container, but first let’s create an arbitrary file containing our desired password:
|
1 |
moore@chom:~$ docker:cloud> echo "secret" > /tmp/mysql_pwd_file |
Now start a container where we’re going to bind mount the file, and use our new environment variable to point to it:
|
1 2 |
moore@chom:~$ docker run -v /tmp/mysql_pwd_file:/tmp/mysqlpwd --name percona-secret -e MYSQL_ROOT_PASSWORD_FILE=/tmp/mysqlpwd percona/percona-server:latest |
With the same inspect command, let’s show that there’s no snooping on our password value:
|
1 2 |
moore@chom:~$ docker inspect --format '{{ index (index .Config.Env) 0}}' percona-secret MYSQL_ROOT_PASSWORD_FILE=/tmp/mysqlpwd |
We are revealing the path where our password was read from within the container. For those eagle-eyed readers, this file was just a bind mounted file in the docker run command, and it’s still on the host’s filesystem.
|
1 2 3 4 5 |
moore@chom:~$ cat /tmp/mysql_pwd_file secret moore@chom:~$ docker exec percona-secret cat /tmp/mysqlpwd secret |
Not perfect, because we need to have that file available on all of our Docker hosts, but it works and we’re closer to a more robust solution.
Docker Secrets
The main reason for the new environment variable is to leverage the docker secrets feature. Since Docker version 1.13 (17.03 is now GA), we have the Docker Secrets feature, however it’s only available to the Docker Swarm workflow. If you’re not already working with Docker Swarm mode, I can’t recommend it enough. It’s part of Docker-engine, simple to get started, and intuitive since 1.13 it is compatible with docker-compose files. You don’t need to have a cluster of hardware, it’s entirely valid to use Docker Swarm on a single node. This allows you to test on your local environment with ease.
I won’t waste pixels explaining what’s already well documented in official channels, but in summary: Docker secrets is a new feature that allows us to keep sensitive information out of source code and configuration files. Secrets are stored in the Raft log which is encrypted and replicated throughout the Docker Swarm cluster. The protection and distribution come for free out of the box, which is a fantastic feature if you ask me.
So, let’s create a Docker Secret. Please note that I’ve moved to my Docker Swarm installation for this next part:
|
1 2 3 |
moore@chom:~$ docker:cloud> docker info | egrep -i 'swarm|version' Server Version: <strong>17.03.0-ce</strong> Swarm: <strong>active</strong> |
Operating as a swarm manager we have the ability to create a new secret to serve as our root user’s password:
|
1 2 |
moore@chom:~$ docker:cloud> echo "{secret_string}" | docker secret create mysql_root_password - ugd8dx0kae9hbyt4opbolukgi |
We can list all of our existing secrets:
|
1 2 3 |
moore@chom:~$ docker:cloud> docker secret ls ID NAME CREATED UPDATED ugd8dx0kae9hbyt4opbolukgi mysql_root_password Less than a second ago Less than a second ago |
Now our secret has been created, it’s obscured from us. We are unable to see it’s value.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
moore@chom:~$ docker secret inspect mysql_root_password [ { "ID": "ugd8dx0kae9hbyt4opbolukgi", "Version": { "Index": 905780 }, "CreatedAt": "2017-04-11T23:33:08.118037434Z", "UpdatedAt": "2017-04-11T23:33:08.118037434Z", "Spec": { "Name": "mysql_root_password" } |