Percona Operator for MySQL, based on Percona XtraDB Cluster (PXC), provides configuration to expose the ProxySQL, HAProxy, and the PXC instances themselves so that you can use Percona Toolkit to connect to MySQL from these endpoints externally. However, you can also create Kubernetes pods to run Percona Toolkit operations there instead if you do not want to expose additional endpoints.
Here’s how to do it.
First, get the PXC pod’s domain name. Here’s a fast way of obtaining it—we can obtain a shell to one of the PXC pods, connect to MySQL, and extract the details from it:
1 2 3 4 5 6 7 8 9 |
# kubectl exec -it cluster1-pxc-0 -c pxc -- bash bash-4.4$ mysql -uroot -p$MYSQL_ROOT_PASSWORD -e "SHOW GLOBAL VARIABLES LIKE 'wsrep_node_incoming_address'" mysql: [Warning] Using a password on the command line interface can be insecure. +-----------------------------+------------------------------------------------------------+ | Variable_name | Value | +-----------------------------+------------------------------------------------------------+ | wsrep_node_incoming_address | cluster1-pxc-0.cluster1-pxc.default.svc.cluster.local:3306 | +-----------------------------+------------------------------------------------------------+ bash-4.4$ exit |
Now that we have the cluster host details, we can create the pod that will run Percona Toolkit. The MySQL administrator credentials are stored in a secret in the PXC operator. We can reuse the credentials by using the same secret in the pod that we will be creating as well. In particular, the pod will connect as the MySQL root user. Percona Toolkit also has a published image in Docker, so we can use that image in the pod’s container.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
# cat pod-pt-mysql-summary.yaml apiVersion: v1 kind: Pod metadata: name: pod-pt-mysql-summary spec: containers: - name: percona-toolkit image: percona/percona-toolkit command: ["pt-mysql-summary"] args: - "--host=cluster1-pxc-0.cluster1-pxc.default.svc.cluster.local" - "--port=3306" - "--user=root" - "--password=$(root)" envFrom: - secretRef: name: cluster1-secrets restartPolicy: Never |
Notice that the restart policy is set to Never so that if the pod runs into a problem, it won’t restart the pod immediately.
Now, let’s create the pod:
1 2 |
# kubectl create -f pod-pt-mysql-summary.yaml pod/pod-pt-mysql-summary created |
After some time, the pod will complete its operations, and then it will be terminated:
1 2 3 4 5 6 7 8 9 |
# kubectl get pods NAME READY STATUS RESTARTS AGE cluster1-haproxy-0 2/2 Running 0 89m cluster1-haproxy-1 2/2 Running 0 86m cluster1-pxc-0 3/3 Running 0 89m cluster1-pxc-1 3/3 Running 0 86m cluster1-pxc-2 3/3 Running 0 85m percona-xtradb-cluster-operator-7c769dcc6d-glnwq 1/1 Running 0 92m pod-pt-mysql-summary 0/1 Completed 0 17s |
Let’s take a look at the pod’s logs.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# kubectl logs pod-pt-mysql-summary mysql: [Warning] Using a password on the command line interface can be insecure. # Percona Toolkit MySQL Summary Report ####################### System time | 2024-08-05 08:11:15 UTC (local TZ: UTC +0000) # Instances ################################################## Port Data Directory Nice OOM Socket ===== ========================== ==== === ====== # MySQL Executable ########################################### # Slave Hosts ################################################ No slaves found # Report On Port 3306 ######################################## User | root@% Time | 2024-08-05 08:11:15 (UTC) Hostname | cluster1-pxc-0 Version | 8.0.35-27.1 Percona XtraDB Cluster (GPL), Release rel27, Revision 84d9464, WSREP version 26.1.4.3 Built On | Linux x86_64 Started | 2024-08-05 06:44 (up 0+01:26:24) Databases | 5 Datadir | /var/lib/mysql/ Processes | 7 connected, 2 running Replication | Is not a slave, has 0 slaves connected Pidfile | /var/lib/mysql/cluster1-pxc-0.pid (does not exist) # Processlist ################################################ *** redacted *** |
You can see from the output above that it was able to run pt-mysql-summary successfully.
Let me provide an additional example. Let’s say you want to use pt-online-schema-change to add a column and add an index to it. Then, my corresponding pod definition would look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
cat pod-pt-online-schema-change.yaml apiVersion: v1 kind: Pod metadata: name: pod-pt-online-schema-change spec: containers: - name: percona-toolkit image: percona/percona-toolkit command: ["pt-online-schema-change"] args: - "--execute" - "--alter=ADD COLUMN testchar varchar(100) default 'test'" - "D=sbtest,t=sbtest1" - "--host=cluster1-pxc-0.cluster1-pxc.default.svc.cluster.local" - "--port=3306" - "--user=root" - "--password=$(root)" envFrom: - secretRef: name: cluster1-secrets restartPolicy: Never |
Then, let’s create the pod:
1 2 |
# kubectl create -f pod-pt-online-schema-change.yaml pod/pod-pt-online-schema-change created |
Then, we can check from the logs and validate if the DDL operation was successful:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# kubectl logs pod-pt-online-schema-change No slaves found. See --recursion-method if host cluster1-pxc-0 has slaves. Not checking slave lag because no slaves were found and --check-slave-lag was not specified. Operation, tries, wait: analyze_table, 10, 1 copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `sbtest`.`sbtest1`... Creating new table... Created new table sbtest._sbtest1_new OK. Altering new table... Altered `sbtest`.`_sbtest1_new` OK. 2024-08-05T08:16:40 Creating triggers... 2024-08-05T08:16:40 Created triggers OK. 2024-08-05T08:16:40 Copying approximately 109653 rows... 2024-08-05T08:16:41 Copied rows OK. 2024-08-05T08:16:41 Analyzing new table... 2024-08-05T08:16:41 Swapping tables... 2024-08-05T08:16:41 Swapped original and new tables OK. 2024-08-05T08:16:41 Dropping old table... 2024-08-05T08:16:41 Dropped old table `sbtest`.`_sbtest1_old` OK. 2024-08-05T08:16:41 Dropping triggers... 2024-08-05T08:16:41 Dropped triggers OK. Successfully altered `sbtest`.`sbtest1`. |
Finally, we can clean up the pod list by deleting the pods:
1 2 3 |
# kubectl delete pod/pod-pt-mysql-summary pod/pod-pt-online-schema-change pod "pod-pt-mysql-summary" deleted pod "pod-pt-online-schema-change" deleted |
In summary, we can use Kubernetes pods to run Percona Toolkit operations so that you do not need to expose an endpoint from the outside to run them.
Percona Toolkit is a collection of advanced open source command-line tools developed and used by the Percona technical staff. These tools are engineered to perform a variety of server and system tasks that are too difficult or complex to perform manually.