EmergencyEMERGENCY? Get 24/7 Help Now!

Serious build and testing automation

 | January 2, 2013 |  Posted In: MySQL


Here at Percona we’ve spent a lot of time improving our development and testing practices. Why? Because constant innovation keeps us ahead and more productive. We want to work smarter, not harder. One of the tools we use is the Jenkins Continuous Integration server.

We use Jenkins pretty heavily to help with out development processes and workflow. The flexibility, modularity and plugin availability are great advantages to Jenkins, and it being free and open source software that we can freely modify and bug fix is immensely valuable (indeed, I now maintain the Jenkins Bazaar plugin).

We currently have:

  • approximately 32 slaves performing builds
  • 151 top level jobs (many of these are matrix jobs)
  • 2,578 jobs (including all jobs in our matrix jobs)
  • 29,124 builds (we auto-expire most builds, so we’ve done many, many more than this)

totalling 109GB of disk usage and a Jenkins master Java process that uses 4GB of resident memory (10GB virt for those keeping score at home).

Our jobs range from automated merging of approved code (and both regression and performance testing of the code before it gets merged to trunk), nightly performance QA runs, building binaries for release and parameterized builds to help developers test their code.

Our slaves are deployed using only about 800 lines of puppet manifests (along with a few templates, scripts etc) and we’ve found that using virt-manager/virsh and KVM to be the most reliable way of running slaves as virtual machines on our physical hardware.

I’ll be giving a talk at the Developer Automation and Continuous Integration miniconf at linux.conf.au 2013 in Canberra about how we got to this point, hopefully helping others improve their development process and automation.

Stewart Smith

Stewart Smith has a deep background in database internals including MySQL, MySQL Cluster, Drizzle, InnoDB and HailDB. he is also one of the founding core developers of the Drizzle database server. He served at Percona from 2011-2014. He is a former Percona employee.

Leave a Reply


Percona’s widely read Percona Database Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.

Besides specific database help, the blog also provides notices on upcoming events and webinars.
Want to get weekly updates listing the latest blog posts? Subscribe to our blog now! Submit your email address below and we’ll send you an update every Friday at 1pm ET.

No, thank you. Please do not ask me again.