I want to present a tool for running a specific version of open source databases in a single instance, replication setups, and Kubernetes. AnyDbVer deploys MySQL/MariaDB/MongoDB/PostgreSQL for testing and experiments.
Docker (or Podman) or dbdeployer (MySQL-Sandbox successor) could also start a specific database version, but such installations are significantly different from production setups.
There is an Ansible playbook with few roles configurable by environment variables.
You may ignore the Ansible layer because the ./anydbver bash script hides it.
It’s not practical to reserve physical servers or virtual machines for all tasks. Frequently you need just a few database instances and the ability to restart it with systemd and check logs with journalctl. AnydbVer spawns one or more Linux Containers managed by Canonical (Ubuntu) LXD, and LXD containers support systemd not using a layered filesystem setup.
Linux Containers is not an emulation layer. From the “host” system, you can see all processes and files created by containers. Thus the performance is very similar to the same database running on a physical server.
LXD setup is relatively easy, but sometimes you may need to run AnydbVer just with a few commands. Vagrant could start an unmodified Ubuntu virtual machine and automatically configure AnydbVer for you.
You can find the required files and setup instructions at https://github.com/ihanick/anydbver.
Clone the https://github.com/ihanick/anydbver repository to setup LXD or start Vagrant.
|
1 |
git clone https://github.com/ihanick/anydbver.git<br>cd anydbver<br>vagrant up<br>vagrant ssh<br>cd anydbver |
Imagine that you need the exact CentOS 7 package version: Percona Server for MySQL 5.6.43-rel84.3:
|
1 |
$ ./anydbver deploy percona-server:5.6.43-rel84.3<br>$ ./anydbver ssh<br>$ mysql<br>mysql> select version(); |
You are not limited to using full version specification. To use the latest matching version, reduce 5.6.43-rel84.3 down to 5.6.43 or even 5.6. To run other databases, replace percona-server with:
Several containers are not consuming a significant amount of resources. Actually five MySQL container instances will consume the same resources as five individual processes running on the same host. The syntax is:
|
1 |
$ ./anydbver deploy <default node definition> node1 <node1 definition> node2 <node2 definition> ... |
For example, run two independent MySQL instances:
$ ./anydbver mysql node1 mysql
SSH access to default node:
|
1 |
$ ./anydbver ssh<br># or<br>$ ./anydbver ssh default |
All other nodes (replace node1 with other node names):
|
1 |
$ ./anydbver ssh node1 |
You may have a server hostname specified. For example let’s run two Percona Server for MySQL instances:
|
1 |
$ ./anydbver deploy ps:5.7 hostname:leader.percona.local node1 ps:5.7 hostname:follower.percona.local<br>$ ./anydbver ssh leader<br>or ./anydbver ssh leader.percona or leader.percona.local<br>[root@leader ~]# mysql --host follower.percona.local --prompt 'h mysql>'<br>follower.percona.local mysql> |
The most interesting part of modern open-source databases is replication. Even active-active replication setups are starting from a single server (leader or master/primary). Start the first node normally and attach additional nodes with master:nodename or leader:nodename. PXC or Galera servers could participate in both synchronous and asynchronous replication. Thus, for Galera clusters, you need galera-master or galera-leader syntax.
Start a 3 node Percona XtraDB cluster (latest 5.7):
|
1 |
./anydbver deploy pxc:5.7 node1 pxc:5.7 galera-master:default node2 pxc:5.7 galera-master:default |
Run master and two async slaves with Percona Server for MySQL and add all three servers to ProxySQL setup:
|
1 |
./anydbver deploy ps:5.7 node1 ps:5.7 master:default node2 ps:5.7 master:default node3 proxysql master:default |
Setup physical replication with slots for Postgresql 12.3:
|
1 |
./anydbver deploy pg:12.3 node1 pg:12.3 master:default |
Make a Mongo replica set named rs0:
|
1 |
./anydbver deploy psmdb replica-set:rs0 node1 psmdb master:default replica-set:rs0 node2 psmdb master:default replica-set:rs0 |
MongoDB sharding setup requires several server types: servers with data (shardsrv), configuration servers (configsrv), and mongos server:
|
1 |
./anydbver deploy <br>psmdb:4.2 replica-set:rs0 shardsrv <br>node1 psmdb:4.2 master:default replica-set:rs0 shardsrv <br>node2 psmdb:4.2 master:default replica-set:rs0 shardsrv <br>node3 psmdb:4.2 configsrv replica-set:cfg0 <br>node4 psmdb:4.2 configsrv replica-set:cfg0 master:node3 <br>node5 psmdb:4.2 configsrv replica-set:cfg0 master:node3 <br>node6 psmdb:4.2 mongos-cfg:cfg0/node3,node4,node5 mongos-shard:rs0/default,node1,node2 |
The fact that we are already using containers (LXD) shouldn’t confuse you. We can still run docker images inside our nodes. Nested containers and Podman makes it possible.
Let’s deploy the default node with Podman and run the Percona Monitoring and Management (PMM) server docker container in it. Percona Server for MySQL 5.7 with PMM client will run on node1:
|
1 |
./anydbver deploy pmm node1 ps:5.7 pmm-client pmm-server:default |
Kubernetes also could utilize nested containers. There are several small Kubernetes distributions: minikube, microk8s, k3s, k0s. The simplest fully functional Kubernetes could be implemented with k3s.
The first k3s LXD container executes the API server and workers. Additional LXD containers could run more workers. Multiple workers are important to run complex HA applications with hostname anti-affinity.
|
1 |
./anydbver deploy k3s node1 k3s-master:default node2 k3s-master:default<br>./anydbver ssh |
On a default node, we can execute kubectl or helm:
|
1 |
$ kubectl get nodes<br>NAME STATUS ROLES AGE VERSION<br>ihanick-node2 Ready <none> 14m v1.19.2+k3s1<br>ihanick-default Ready master 17m v1.19.2+k3s1<br>ihanick-node1 Ready <none> 15m v1.19.2+k3s1 |
The Kubernetes cluster allows running stateful applications, like databases, services (similar to Amazon AWS S3), or monitoring solutions. Let’s start:
|
1 |
./anydbver deploy k3s <br>node1 k3s-master:default <br>node2 k3s-master:default <br>node3 k3s-master:default <br>default k8s-minio k8s-pmm k8s-pxc backup<br>./anydbver ssh<br>kubectl get pods<br>NAME READY STATUS RESTARTS AGE<br>svclb-monitoring-service-n5nsx 1/1 Running 0 20m<br>svclb-monitoring-service-htssw 1/1 Running 0 20m<br>svclb-monitoring-service-n9kt4 1/1 Running 0 20m<br>svclb-monitoring-service-7btbh 1/1 Running 0 20m<br>minio-service-6db6667fb9-tk69n 1/1 Running 0 20m<br>monitoring-0 1/1 Running 0 20m<br>percona-xtradb-cluster-operator-7886ccf6b5-rtwxc 1/1 Running 0 18m<br>cluster1-pxc-0 2/2 Running 2 17m<br>cluster1-haproxy-0 2/2 Running 0 17m<br>cluster1-haproxy-1 2/2 Running 0 12m<br>cluster1-haproxy-2 2/2 Running 0 12m<br>cluster1-pxc-1 2/2 Running 1 13m<br>cluster1-pxc-2 2/2 Running 1 10m |
You can install and configure the database in the same way, regardless of the architecture:
|
1 |
./anydbver deploy k3s <br> node1 k3s-master:default <br> node2 k3s-master:default <br> node3 k3s-master:default <br> default k8s-minio k8s-mongo backup |
You are not limited to Percona-only products and can run other K8s operators:
|
1 |
./anydbver deploy k3s node1 k3s-master:default node2 k3s-master:default node3 k3s-master:default default k8s-pg |
AnyDbVer is a useful deployment tool for experiments with:
AnyDbVer restrictions:
Resources
RELATED POSTS
