October 23, 2014

New patches, new builds

We made new patches, improved previous and want to announce new builds for 5.0.62, 5.0.67 and 5.1.26 versions. One of biggest changes we separated releases of 5.0 into two branches.
First, just “-percona” release is more stable and contains only stable and proven on many installation patches. Second is “-percona-highperf” release, which contains experimental patches that can improve performance significantly. On this stage “percona-highperf” contains patches from Yasufumi Kinoshita (more info on Maximum performance of OLTP benchmark is not so scalable on multi-cpu.
Most important patch “split_buf_pool_mutex” to separate InnoDB buffer->pool mutex into several, and it allows to get 1.5x – 2.5x improvements in DBT2 benchmarks on 8cores boxes (benchmark results is coming and will be published)

For all three versions we added SHOW PATCHES command, which is based on Jeremy Cole’s patch

Next, we added tables USER_STATISTICS, INDEX_STATISTICS, TABLE_STATISTICS to INFORMATION_SCHEMA

For 5.0-percona-highperf release we added:

  • split_buf_pool_mutex_fixed_optimistic_safe.patch | InnoDB patch to fix buffer pool scalability
  • control_io-threads.patch | InnoDB patch to control count of IO threads
  • control_flush_and_merge_and_read.patch | InnoDB patch to control insert buffer and flushing

5.1 release
Contains build-in InnoDB plugin, so there is no more needs to download InnoDB separately, install it and run mysql with option –skip-innodb.
We think InnoDB plugin is stable enough to use it with MySQL 5.1 RC, at least stable to name whole MySQL-5.1-percona release as RC.

Along with previously mentioned patches, to 5.1 we added new column TIME_MS to
INFORMATION_SCHEMA.PROCESSLIST which contains time of query execution with microseconds granularity.

On our release page
you can find links to RPMS for RedHat 4/5, binary tar.gz, source tar.gz (source contains source code with applied patches, so you can just download it and compile) and just raw patches.

About Vadim Tkachenko

Vadim leads Percona's development group, which produces Percona Clould Tools, the Percona Server, Percona XraDB Cluster and Percona XtraBackup. He is an expert in solid-state storage, and has helped many hardware and software providers succeed in the MySQL market.

Comments

  1. Great stuff! I’ve seen Yasufumi’s, and I’ve been meaning to have a proper look at them for some time now, but never really gotten around to actually doing it. Certainly will now!
    Thanks a bunch for all the hard work you guys put in! Keep this up, and your release page will be the default for obtaining MySQL rather than dev.mysql.com/downloads :)

    This is in my opinion the way MySQL GA should be developed!

  2. This is great. Thanks for sharing and helping to make our 8+ core servers somewhat useful.

  3. MSDI says:

    Hi, sorry for the dumb questions, but since these patchs add useful functionnalities and performance improvements, why are they not included in the regular mysql build ? Is it because of licensing or ?

    Thanks

  4. Vadim says:

    MSDI,

    It is more question to MySQL / Sun :)

    The standard answer is the MySQL 5.0 / 5.1 is frozen for new features, and they can be included only in MySQL 6.0.

    But we are working with real customers who use 5.0 mostly and we need our features right now, not in 3-5 years.

    I think this is one of reasons why Drizzle was forked – to have possibility to easy extend.

  5. jfbus says:

    Why not include SphinxSE ?

  6. ngm says:

    While trying to rebuild MySQL-percona-3-5.0.67-0.rhel5.src.rpm on a i386 architecture (CentOS 5.2), it complains about y.tab.c not found.

  7. @ngm, make sure you’ve got ‘bison’ installed.

  8. Vadim says:

    jfbus,

    I think SphinxSE is not so popular yet…
    But we look into

  9. For some reason I didn’t see this before.

    I think we’re going to look at these as we’re looking at our 5.0.x migration (just now I know) so this is good timing to get these patches in…..

  10. peter says:

    Kevin,

    I also should note this effort is driven both by customers and community. When our customers need something we get it included in our version, plus we add some things we think is generally helpful even though there is no customer sponsorship for it.

    So if you have some Innodb issues – Ie I remember you would like buffer flush to work doing more sequential IO you can let us know and we might be able to fix thing for your particular workload.

  11. Vladimir says:

    А будут ли подобные сборки для Debian?
    Хотелось бы потестировать “-percona-highperf” и “-percona” на 8-ми ядрах под обе операционки.

  12. Vladimir says:

    Hi.
    Vadim, do you planing to make same builds for debian_amd64 system, both “-percona” and “-percona-highperf” branches ?
    I’d realy like to test my 8cores box :-)

  13. Vadim says:

    Vladimir,

    We have plans to make builds for Debian, but currently have no Debian boxes, so I can’t say when we will build that.

  14. Vladimir,

    If it helps we can give you access to a Debian box if you ship dpkgs :)

    Kevin

  15. Vladimir says:

    Kevin,

    ..do you mean by Vadim?

    Thanks anyway :-).
    I’m already have a couple of Debian boxes :-), so I can try to make dpkgs.

  16. Hi,

    was about to create Mysql-highperf RPM on my CentOS 5.1. Had such problem:
    [root@record mysql_highperf]# ll
    -rw-r–r– 1 root root 28308310 Sep 9 03:04 MySQL-percona-highperf-3-5.0.68-0.rhel5.src.rpm
    [root@record mysql_highperf]# rpm -ihv MySQL-percona-highperf-3-5.0.68-0.rhel5.src.rpm
    1:MySQL-percona-highperf-########################################### [100%]
    [root@record mysql_highperf]# rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec
    error: parse error in expression
    error: /usr/src/redhat/SPECS/mysql-percona.rhel5.spec:103: parseExpressionBoolean returns -1
    error: Name field must be present in package: (main package)
    error: Version field must be present in package: (main package)
    error: Release field must be present in package: (main package)
    error: Summary field must be present in package: (main package)
    error: Group field must be present in package: (main package)
    error: License field must be present in package: (main package)

    tar and rpm related packages are up to date. Perhaps you could let us know what to do :) Versions:
    rpm-python-4.4.2-48.el5
    rpm-libs-4.4.2-48.el5
    rpm-build-4.4.2-48.el5
    rpm-4.4.2-48.el5
    tar-1.15.1-23.0.1.el5

    Cheers, Askar

  17. Vadim says:

    Askar,

    Please use
    rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘percona 1′ –define ‘mysqlversion 5.0.68′
    for usual release and
    rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘perconahighperf 1′ –define ‘mysqlversion 5.0.68′
    for HighPerf release

  18. Sheeri says:

    The “binary” versions should work on Debian, right? (just noting Kevin’s question….the “binary” versions will run on most linux versions — just like the MySQL binaries, right?)

    Also, what about Solaris? It’s not really clear what the binaries are for….

  19. Askar says:

    Preved Vadim,

    thanks for prompt reply, it helped!

    However, either source code or SRPM can not compile. It says:

    Making all in sql
    make[2]: Entering directory /usr/src/redhat/BUILD/mysql-5.0.68/sql'
    d --debug --verbose sql_yacc.yy
    make[2]: d: Command not found
    make gen_lex_hash
    make[2]: [sql_yacc.cc] Error 127 (ignored)
    if test -f y.tab.h; then \
    to=
    echo “sql_yacc_H” | sed \
    -e ‘y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/’ \
    -e ‘s/[^ABCDEFGHIJKLMNOPQRSTUVWXYZ]/_/g’; \
    sed -e "/^#/!b" -e "s/Y_TAB_H/$to/g" -e "s|y\.tab\.h|sql_yacc.h|" \
    y.tab.h >sql_yacc.ht; \
    rm -f y.tab.h; \
    if cmp -s sql_yacc.ht sql_yacc.h; then \
    rm -f sql_yacc.ht ;\
    else \
    mv sql_yacc.ht sql_yacc.h; \
    fi; \
    fi
    /usr/bin/perl patch_info.h.pl > patch_info.h
    if test -f y.output; then \
    mv y.output sql_yacc.output; \
    fi
    sed '/^#/ s|y\.tab\.c|sql_yacc.cc|' y.tab.c >sql_yacc.cct && mv sql_yacc.cct sql_yacc.cc
    sed: can't read y.tab.c: No such file or directory
    make[2]: *** [sql_yacc.cc] Error 2
    make[2]: *** Waiting for unfinished jobs....
    make[3]: Entering directory
    /usr/src/redhat/BUILD/mysql-5.0.68/sql’
    if gcc -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME=”\”/\”” -DDATADIR=”\”/var/lib/mysql\”” -DSHAREDIR=”\”/usr/share/mysql\”” -DHAVE_CONFIG_H -I. -I. -I../include -I../zlib -I../innobase/include -I../innobase/include -I../include -I../include -I../regex -I. -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -MT gen_lex_hash.o -MD -MP -MF “.deps/gen_lex_hash.Tpo” -c -o gen_lex_hash.o gen_lex_hash.cc; \
    then mv -f “.deps/gen_lex_hash.Tpo” “.deps/gen_lex_hash.Po”; else rm -f “.deps/gen_lex_hash.Tpo”; exit 1; fi
    -mcpu=' is deprecated. Use -mtune=’ or ‘-march=’ instead.
    /bin/sh ../libtool –preserve-dup-deps –tag=CXX –mode=link gcc -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -rdynamic -o gen_lex_hash -static gen_lex_hash.o ../myisam/libmyisam.a ../myisammrg/libmyisammrg.a ../heap/libheap.a ../vio/libvio.a ../mysys/libmysys.a ../dbug/libdbug.a ../regex/libregex.a ../strings/libmystrings.a ../zlib/libzlt.la -lpthread -lcrypt -lnsl -lm -lpthread
    mkdir .libs
    gcc -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -rdynamic -o gen_lex_hash gen_lex_hash.o ../myisam/libmyisam.a ../myisammrg/libmyisammrg.a ../heap/libheap.a ../vio/libvio.a ../mysys/libmysys.a ../dbug/libdbug.a ../regex/libregex.a ../strings/libmystrings.a ../zlib/.libs/libzlt.a -lpthread -lcrypt -lnsl -lm -lpthread
    make[3]: Leaving directory /usr/src/redhat/BUILD/mysql-5.0.68/sql'
    ./gen_lex_hash > lex_hash.h-t
    /bin/mv lex_hash.h-t lex_hash.h
    make[2]: Leaving directory
    /usr/src/redhat/BUILD/mysql-5.0.68/sql’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/usr/src/redhat/BUILD/mysql-5.0.68′
    make: *** [all] Error 2
    error: Bad exit status from /var/tmp/rpm-tmp.97483 (%build)

    RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.97483 (%build)

    Hope you could somehow help?

    Cheers, Askar

  20. Vadim says:

    Askar,

    You probably need to install bison and/or flex

  21. Vadim says:

    Sheeri,

    “Binary” should work on Debian, though we did not test.
    It is Linux glibc2.3 binary, we will fix description, thank you.

    There are no Solaris binaries and no plans for, unless there are requests for.

  22. Sid says:

    Hi,

    Have you compared the, “split_buf_pool_mutex” patch with Google’s smpfix patch? It sounds like the two patches are incompatible. How do they compare in stability and performance?

  23. Vadim says:

    Sid,

    It is two different patches and they solve different problems.
    We are going to publish our improved version of “smpfix”, which is compatible with our patches.

  24. Lee says:

    i have tried rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘percona 1′ –define ‘mysqlversion 5.0.68′, but there’s an error:
    -define: unknown option

    Please help. Thanks

Speak Your Mind

*