As we continue the development of our Percona Operators to simplify database deployment in Kubernetes (Percona Server for MongoDB Operator 0.3.0 and Percona XtraDB Cluster Operator 0.3.0), one very popular question I get is: how does deployment in Kubernetes affect MySQL performance? Is there a big performance penalty? So I plan to look at how to measure and compare the performance of Kubernetes deployments to bare metal deployments. Kubernetes manages a lot of infrastructure resources like network, storage, cpu, and memory, so we need to look individually at different components.
To begin: I plan to run a single MySQL (Percona Server) instances in a Kubernetes deployment, and use local storage (fast NMVe device). I also want to customize my MySQL configuration, as the one that is supplied in public images are pretty much all set to defaults.
Let’s take a look at how we can customize it.
Here’s an example yaml file:
|
1 |
=====================<br>apiVersion: v1<br>kind: Service<br>metadata:<br> name: mysql<br>spec:<br> ports:<br> - port: 3306<br> selector:<br> app: mysql<br> clusterIP: None<br>---<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: mysql<br> spec:<br> selector:<br> matchLabels:<br> app: mysql<br> strategy:<br> type: Recreate<br> template:<br> metadata:<br> labels:<br> app: mysql<br> spec:<br> nodeSelector:<br> kubernetes.io/hostname: smblade01<br> volumes:<br> - name: mysql-persistent-storage<br> hostPath:<br> path: /mnt/fast/mysql<br> type: Directory<br> containers:<br> - image: percona:ps-8.0.15-5<br> name: mysql<br> env:<br> # Use secret in real usage<br> - name: MYSQL_ROOT_PASSWORD<br> value: password<br> ports:<br> - containerPort: 3306<br> name: mysql<br> volumeMounts:<br> - name: mysql-persistent-storage<br> mountPath: /var/lib/mysql<br>=============== |
There is a lot of typical Kubernetes boilerplate to create a deployment, but the most important parts to note:
After deploying this, we make sure the Pod is running
|
1 |
Kubectl get pods<br>NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES<br>mysql-d74d5d459-b6zrs 1/1 Running 0 3m8s 192.168.1.15 smblade01 <none> <none> |
Now as we are running MySQL on a dedicated node with fast storage, we want to customize the MySQL configuration to allocate a big buffer pool and adjust its IO settings.
As I said, a downloaded image will most likely run with default settings and there is no straightforward way to pass our custom my.cnf to deployment. I’ll show you how to resolve this now.
The default my.cnf contains the directive !includedir /etc/my.cnf.d
So the solution for the custom my.cnf is as follows:
|
1 |
kubectl create configmap mysql-config --from-file=my.cnf |
|
1 |
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2<br>kind: Deployment<br>metadata:<br> name: mysql<br>spec:<br> selector:<br> matchLabels:<br> app: mysql<br> strategy:<br> type: Recreate<br> template:<br> metadata:<br> labels:<br> app: mysql<br> spec:<br> nodeSelector:<br> kubernetes.io/hostname: smblade01<br> volumes:<br> - name: mysql-persistent-storage<br> hostPath:<br> path: /mnt/fast/mysql<br> type: Directory<br> - name: config-volume<br> configMap:<br> name: mysql-config<br> optional: true<br> containers:<br> - image: percona:ps-8<br> name: mysql<br> env:<br> # Use secret in real usage<br> - name: MYSQL_ROOT_PASSWORD<br> value: password<br> ports:<br> - containerPort: 3306<br> name: mysql<br> volumeMounts:<br> - name: mysql-persistent-storage<br> mountPath: /var/lib/mysql<br> - name: config-volume<br> mountPath: /etc/my.cnf.d<br> |
And here’s our example my.cnf:
|
1 |
[mysqld]<br><br>skip-log-bin<br>ssl=0<br><br>table_open_cache = 200000<br>table_open_cache_instances=64<br>back_log=3500<br>max_connections=4000<br><br>innodb_file_per_table<br><br>innodb_log_file_size=10G<br>innodb_log_files_in_group=2<br>innodb_log_buffer_size=64M<br><br>innodb_open_files=4000<br><br>innodb_buffer_pool_size= 100G<br>innodb_buffer_pool_instances=8<br><br>innodb_flush_log_at_trx_commit = 1<br>innodb_doublewrite=1<br>innodb_flush_method = O_DIRECT<br>innodb_file_per_table = 1<br>innodb_io_capacity=2000<br>innodb_io_capacity_max=4000<br>innodb_flush_neighbors = 0<br>innodb_use_native_aio=1<br><br>join_buffer_size=256K<br>sort_buffer_size=256K<br> |
When we deploy this yaml, we will have a MySQL instance running on a dedicated box with fast local storage, big log files, and 100GB allocated for its InnoDB buffer pool.
Now we’re set to proceed to our performance measurements. Stay tuned!
—
Photo by Joseph Barrientos on Unsplash
Resources
RELATED POSTS