EmergencyEMERGENCY? Get 24/7 Help Now!

Benchmarks challenges of XtraDB Cluster


Posted on:

|

By:


PREVIOUS POST
NEXT POST
Share Button

We are running internally a lot of benchmarks on our recently announced Percona XtraDB Cluster, and I am going to publish these results soon.
But before that I wanted to mention that proper benchmark of distributed system comes with a lot of challenges.
I am saying that not to complain, but to make sure, if you are going to benchmark XtraDB Cluster yourself, there is a lot of things to take into account.

And it seems that one component, which was not much important before, now appears as critical peace, which easily can became bottleneck in the benchmarks – this is network.

In case of simple client-server setup, the network is not fully utilized.

But as we start testing a cluster setup, the 1Gb network between client and switch is getting fully utilized by sysbench communication with 3 nodes.

In this setup it does not make sense to increase number of nodes, as we will not be able to load them properly.

The solution would be to increase network capacity or add additional client boxes.

Now take into account that there is an internal network communication between nodes also, and that makes a network tuning as the critical part of a cluster setup. This is not something we paid much attention before.

The main conclusion of this post is that if you are going to benchmark a Percona XtraDB Cluster or just use it under intensive communication workload, pay an attention to network component. It is very easy that a client or a client network becomes bottleneck.


Share Button
PREVIOUS POST
NEXT POST


Vadim Tkachenko

Vadim leads Percona's development group, which produces the Percona Server, Percona Server for MongoDB, Percona XtraDB Cluster and Percona XtraBackup. He is an expert in solid-state storage, and has helped many hardware and software providers succeed in the MySQL market.



Categories:
Benchmarks, MySQL, XtraDB Cluster


Comments
  • Maybe use a dedicated haproxy server with incoming sysbench traffic coming in on eth0 and load balance to the Galera cluster on eth1.

    Load balancer can do weighted balancing to simulate more real world transaction distribution. You can listen on multiple ports/ips to vary distribution of writes/reads, slow queries/fast queries, etc.

    Reply

  • Vadim,

    Yeah. You can do standard benchmarks basically having single server. Cluster benchmarks require proper lab with network setup correctly etc which also adds a lot more variance to results. Using multiple clients for benchmark is the most realistic as this is what we frequently have – tens of web servers can be talking to single MySQL server or cluster.

    Reply

  • How difficult is to bring InfiniBand into play? Not IP-over-InfiniBand, but native? I bet this would be great for cluster nodes located into same rack or same datacenter.

    Reply

  • I would be interested to see if IPv4 or IPv6 makes a difference in the benchmarks. And of course MTU size and jumbo frames could be a factor. What about using network bonding or nic teaming (new in linux 3.3?)? This could provide more bandwidth if there are enough network ports available.

    Reply

  • HI,

    Today I installed percona xtradb cluster on three of my laptops for testing, laptops connect with wireless router. I immediately realised that there are latency issues, then I moved to 100 mbps Ethernet connection. Quality of cables and the router is not giving more then 9-10 Mbps max.

    I was able to connect the nodes together and then played with them they connect fine, but now comes the main part. I have a database script that populates my database of 1GB, once it starts filling the tables with data it crashes, I mean the cluster crashes and then I have to do manual recovery of the cluster.
    Now I am wondering if it is because of my network is not good enough for such testing or is it something to do with cluster limitations that it cannot handle so many writes.

    Would appreciate your answers…
    regards,
    Alam

    Reply

Leave a Reply

Percona’s widely read Percona Data Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.

Besides specific database help, the blog also provides notices on upcoming events and webinars.

Want to get weekly updates listing the latest blog posts? Subscribe to our blog now! Submit your email address below.

No, thank you. Please do not ask me again.