Percona Playback 0.3 development release

I’m glad to announce the third Percona Playback release – another alpha release of a new software package designed to replay database server load. The first two versions were released in April, just in time for my talk at the Percona Live MySQL Conference and Expo: Replaying Database Load with Percona Playback.

This is still very much under development, so there’s likely going to be bugs. Please feel free to report bugs here:

Percona Playback is designed to replay database load captured either in a MySQL slow query log or a tcpdump capture of the MySQL protocol exchange between client and server.

It can replay the load either as fast as possible or in accurate mode, where it tries to replay load over the same wall time as the capture.

Current Notable Limitations:

  • tcpdump replay: IPv4 only
  • tcpdump replay: no support for server side prepared statements

Build requirements:

  • libtbb-dev (Intel Threading Building blocks)
  • boost (including boost program_options)
  • intltool
  • gettext
  • libpcap-dev
  • libcloog-ppl (if using gcc 4.6)
  • libmysqlclient-dev
  • libdrizzle-dev
  • pkg-config

Source Tarball: percona-playback-0.3.tar.gz (md5sig)

We’ve tested building on CentOS/RHEL 6, Ubuntu 10.04LTS (lucid), Ubuntu 12.04 (precise) and Debian 6.

You can build using the standard: ./configure && make && make install

Run the test suite: make check

We are planning binary packages for the next development milestone.

Share this post

Comments (17)

  • westxi

    Can you show the steps in rh6?We met some wrong dependencies on it
    Thank you

    July 11, 2012 at 3:50 am
  • Stewart Smith

    You’ll need at least the following packages (some are from EPEL):

    yum install libpcap-devel pkgconfig libdrizzle-devel gettext-devel intltool boost-devel boost-program-options boost-static boost-thread boost tbb-devel mysql-devel

    There may be a couple more, but this is what I found from glancing at our puppet config. Does this help?

    July 11, 2012 at 4:03 am
  • weixi

    We have test tcpcopy&tcpreplay for a month to archive real flow benchmark.
    Now found playback is readlly good tool.
    We run it successfully on ubuntu,But on our product environment redhat 6u1,not yet succeeded
    Can you provide yum source or playback.el6.rpm to us for further testing


    July 11, 2012 at 8:35 pm
  • Stewart Smith

    Great to hear! I’d be interested to hear how accurate you’re finding the reproduced workload compared to the original, especially compared to tcpcopy/tcpreplay.

    We’re planning to have binary packages with the next development snapshot, 0.4, which is probably 1-2months away.

    I’d love to see what errors you’re getting trying to build on RHEL6, are you able to post here or on launchpad answers?

    July 11, 2012 at 9:02 pm
  • weixi

    sudo yum install libpcap-devel pkgconfig libdrizzle-devel gettext-devel intltool boost-devel boost-program-options boost-static boost-thread boost tbb-devel 2> /dev/null |grep ‘No package’
    No package libpcap-devel available.
    No package libdrizzle-devel available.
    No package boost-static available.
    No package tbb-devel available.

    Four package’s dependencies

    July 12, 2012 at 6:10 am
  • Martin


    Those package are in the Epel repository.

    For example:
    # yum info libdrizzle-devel
    Name : libdrizzle-devel
    From repo : epel
    Description : Development files for the Drizzle Client & Protocol Library


    July 12, 2012 at 9:20 am
  • weixi

    Well Done
    Thank you very much:)

    July 12, 2012 at 10:07 am
  • Stewart Smith

    Yep, it’s in the EPEL repository. It’d be great if RH/CentOS shipped these in main (hint hint to anybody listening)

    We may also add them to the Percona repository when we ship binaries of Percona Playback, as this would certainly make installation easier.

    July 12, 2012 at 4:57 pm
  • weixi

    We meet a small problem, Use tcpdump capture multiple ip traffic on the 3306, playback return some errors, the pressure will be reduced.

    July 14, 2012 at 12:31 am
  • weixi

    Hi Smith

    we found PB can only replay QPS/TPS in test database,that‘s why we can get high qps and tps on mysql server ,but none on innodb layer.
    Please check the code


    July 17, 2012 at 7:49 am
  • Stewart Smith

    Ahh… and your load is across several databases? That could be an issue with the current code for sure.

    July 19, 2012 at 4:46 pm
  • sh

    Interesting tool, it looks promising. I couldn’t make it work, though. Can it read general query logs? I tried to run it with a general log, but it failed on each line with an “Error query: …” error. In addition, it loaded the whole log file (sized 15GB) into memory, consuming totally about 18GB of memory which was a little unexpected.

    There’s not much point in replaying only the slow logs, because they don’t normally represent a normal load pattern (ideally, the slow log must have zero records at all). Replaying general logs is much more interesting. The tcpdump file replay feature could look like a possible workaround, but from the previous comments it seems that percona-playback cannot play a tcpdump file having more than one client (src IP) recorded. This makes it impossible to capture and replay a production server’s workload without utilizing some kind of proxy in front of the MySQL server.

    Also, is there a way to set the number of replay execution threads (concurrent DB connections)? I couldn’t find such an option.

    August 12, 2012 at 3:35 pm
  • Stewart Smith

    No, it doesn’t currently replay the general query log. The slow query log can capture all queries if the slow query time is set to zero.

    The loading the entire file into memory is likely a bug on invalid input, if you write up a bug report and attach a sample file, you’ll be the best person in the entire universe 🙂 (we should certainly fix that bug).

    It should work with more than one source ip recorded.

    Replay execution threads are currently one per DB connection (as in, 1000 connections to database = 1000 threads) as this is what we must do to work with libmysqlclient. We’re planning to have a future release that will use a non blocking client library (libdrizzle) and a pool of threads.

    August 12, 2012 at 5:32 pm
  • sh

    That makes sense, thank you. I am now capturing a tcpdump file on a live server with multiple clients accessing different databases and will try to replay it on test hardware. I forced every client to reconnect to the DB shortly after tcpdump was started to make sure that every session is recorded from its very start (we utilize persistent connections, so I guess they would be recorded starting in the middle otherwise).

    I will submit all bugs I find as soon as I’m done (but there’s no way I can upload a 15GB production query log file :)).

    August 13, 2012 at 6:22 am
  • Stewart Smith

    Hopefully we’ll be able to reproduce problems that you describe, but many thanks for giving it a go and I look forward to hearing how it went!

    August 13, 2012 at 9:12 pm
  • Dave Hentchel

    In our field work we find that customers more often have General logs available, rather than usable Slow query logs. Is there anything inherently difficult in supporting general logs for this tool, or is it just a matter of adding an input plugin that knows how to parse them?

    September 25, 2013 at 11:47 am
  • Stewart Smith

    You could add support to parse the general log and that would be fine (and it shouldn’t be too hard and I welcome your patch! – Please email me if you have any queries on implementing it)… I can’t currently think of any real barriers to it working okay but naturally there’s going to be some limitations, many of them shared with the slow query log.

    September 25, 2013 at 6:43 pm

Comments are closed.

Use Percona's Technical Forum to ask any follow-up questions on this blog topic.