We made new patches, improved previous and want to announce new builds for 5.0.62, 5.0.67 and 5.1.26 versions. One of biggest changes we separated releases of 5.0 into two branches.
First, just “-percona” release is more stable and contains only stable and proven on many installation patches. Second is “-percona-highperf” release, which contains experimental patches that can improve performance significantly. On this stage “percona-highperf” contains patches from Yasufumi Kinoshita (more info on Maximum performance of OLTP benchmark is not so scalable on multi-cpu.
Most important patch “split_buf_pool_mutex” to separate InnoDB buffer->pool mutex into several, and it allows to get 1.5x – 2.5x improvements in DBT2 benchmarks on 8cores boxes (benchmark results is coming and will be published)
For all three versions we added SHOW PATCHES command, which is based on Jeremy Cole’s patch
|
1 2 3 4 5 6 7 8 9 10 11 12 |
mysql> show patches; +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+ | File | Name | Version | Author | License | Comment | +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+ | control_flush_and_merge_and_read.patch | InnoDB patch to control insert buffer and flushing | 1.0 | Yasufumi Kinoshita | BSD | | | mysqld_safe_syslog.patch | Patch allows redirect output of error.log to syslog-ng | 1.0 | Percona <info@percona.com> | GPL | Ported from Debian | | microslow_innodb.patch | Extended statistics in slow.log | 1.0 | Percona <info@percona.com> | GPL | | | split_buf_pool_mutex_fixed_optimistic_safe.patch | InnoDB patch to fix buffer pool scalability | 1.0 | Yasufumi Kinoshita | BSD | | | control_io-threads.patch | InnoDB patch to control count of IO threads | 1.0 | Yasufumi Kinoshita | BSD | | | userstats.patch | SHOW USER/TABLE/INDEX statistics | 1.0 | Google | GPL | Added INFORMATION_SCHEMA.*_STATISTICS | | show_patches.patch | SHOW PATCHES | 1.0 | Jeremy Cole | N/A | | +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+ |
Next, we added tables USER_STATISTICS, INDEX_STATISTICS, TABLE_STATISTICS to INFORMATION_SCHEMA
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
mysql> select * from information_schema.table_statistics; +-------------------------+-----------+--------------+----------------------+ | TABLE_NAME | ROWS_READ | ROWS_CHANGED | ROWS_CHANGED_INDEXES | +-------------------------+-----------+--------------+----------------------+ | art86.forum86 | 469084 | 54 | 162 | | mysql.db | 36 | 0 | 0 | | art57.link_out57 | 59336163 | 82549 | 577843 | | art116.author116 | 1921596 | 1581 | 3162 | | art54.link_out54 | 36879450 | 23830 | 166810 | | art104.author104 | 1100203 | 2572 | 5144 | | art107.forum107 | 820296 | 56 | 168 | | art104.forum104 | 771121 | 63 | 189 | | art111.article111 | 44100332 | 16411 | 164110 | | art93.thread_stats93 | 442440 | 9495 | 18990 | ... |
For 5.0-percona-highperf release we added:
- split_buf_pool_mutex_fixed_optimistic_safe.patch | InnoDB patch to fix buffer pool scalability
- control_io-threads.patch | InnoDB patch to control count of IO threads
- control_flush_and_merge_and_read.patch | InnoDB patch to control insert buffer and flushing
5.1 release
Contains build-in InnoDB plugin, so there is no more needs to download InnoDB separately, install it and run mysql with option –skip-innodb.
We think InnoDB plugin is stable enough to use it with MySQL 5.1 RC, at least stable to name whole MySQL-5.1-percona release as RC.
Along with previously mentioned patches, to 5.1 we added new column TIME_MS to
INFORMATION_SCHEMA.PROCESSLIST which contains time of query execution with microseconds granularity.
|
1 2 3 4 5 6 7 |
mysql> SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST; +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+ | ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO | TIME_MS | +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+ | 1 | root | localhost | NULL | Query | 0 | executing | SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST | 0.607 | +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+ 1 row in set (0.00 sec) |
On our release page
you can find links to RPMS for RedHat 4/5, binary tar.gz, source tar.gz (source contains source code with applied patches, so you can just download it and compile) and just raw patches.
About the Author
Vadim TkachenkoVadim Tkachenko co-founded Percona in 2006 and serves as its Chief Technology Officer. Vadim leads Percona Labs, which focuses on technology research and performance evaluations of Percona’s and third-party products. Vadim’s expertise in LAMP performance and multi-threaded programming help optimize MySQL and InnoDB internals to take full advantage of modern hardware. He also co-authored the book High Performance MySQL: Optimization, Backups, and Replication 3rd Edition.
Great stuff! I’ve seen Yasufumi’s, and I’ve been meaning to have a proper look at them for some time now, but never really gotten around to actually doing it. Certainly will now!
Thanks a bunch for all the hard work you guys put in! Keep this up, and your release page will be the default for obtaining MySQL rather than dev.mysql.com/downloads 🙂
This is in my opinion the way MySQL GA should be developed!
This is great. Thanks for sharing and helping to make our 8+ core servers somewhat useful.
Hi, sorry for the dumb questions, but since these patchs add useful functionnalities and performance improvements, why are they not included in the regular mysql build ? Is it because of licensing or ?
Thanks
MSDI,
It is more question to MySQL / Sun 🙂
The standard answer is the MySQL 5.0 / 5.1 is frozen for new features, and they can be included only in MySQL 6.0.
But we are working with real customers who use 5.0 mostly and we need our features right now, not in 3-5 years.
I think this is one of reasons why Drizzle was forked – to have possibility to easy extend.
Why not include SphinxSE ?
While trying to rebuild MySQL-percona-3-5.0.67-0.rhel5.src.rpm on a i386 architecture (CentOS 5.2), it complains about y.tab.c not found.
@ngm, make sure you’ve got ‘bison’ installed.
jfbus,
I think SphinxSE is not so popular yet…
But we look into
For some reason I didn’t see this before.
I think we’re going to look at these as we’re looking at our 5.0.x migration (just now I know) so this is good timing to get these patches in…..
Kevin,
I also should note this effort is driven both by customers and community. When our customers need something we get it included in our version, plus we add some things we think is generally helpful even though there is no customer sponsorship for it.
So if you have some Innodb issues – Ie I remember you would like buffer flush to work doing more sequential IO you can let us know and we might be able to fix thing for your particular workload.
Рбудут ли подобные Ñборки Ð´Ð»Ñ Debian?
ХотелоÑÑŒ бы потеÑтировать “-percona-highperf” и “-percona” на 8-ми Ñдрах под обе операционки.
Hi.
Vadim, do you planing to make same builds for debian_amd64 system, both “-percona” and “-percona-highperf” branches ?
I’d realy like to test my 8cores box 🙂
Vladimir,
We have plans to make builds for Debian, but currently have no Debian boxes, so I can’t say when we will build that.
Vladimir,
If it helps we can give you access to a Debian box if you ship dpkgs 🙂
Kevin
Kevin,
..do you mean by Vadim?
Thanks anyway :-).
I’m already have a couple of Debian boxes :-), so I can try to make dpkgs.
Hi,
was about to create Mysql-highperf RPM on my CentOS 5.1. Had such problem:
[root@record mysql_highperf]# ll
-rw-r–r– 1 root root 28308310 Sep 9 03:04 MySQL-percona-highperf-3-5.0.68-0.rhel5.src.rpm
[root@record mysql_highperf]# rpm -ihv MySQL-percona-highperf-3-5.0.68-0.rhel5.src.rpm
1:MySQL-percona-highperf-########################################### [100%]
[root@record mysql_highperf]# rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec
error: parse error in expression
error: /usr/src/redhat/SPECS/mysql-percona.rhel5.spec:103: parseExpressionBoolean returns -1
error: Name field must be present in package: (main package)
error: Version field must be present in package: (main package)
error: Release field must be present in package: (main package)
error: Summary field must be present in package: (main package)
error: Group field must be present in package: (main package)
error: License field must be present in package: (main package)
tar and rpm related packages are up to date. Perhaps you could let us know what to do 🙂 Versions:
rpm-python-4.4.2-48.el5
rpm-libs-4.4.2-48.el5
rpm-build-4.4.2-48.el5
rpm-4.4.2-48.el5
tar-1.15.1-23.0.1.el5
Cheers, Askar
Askar,
Please use
rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘percona 1’ –define ‘mysqlversion 5.0.68’
for usual release and
rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘perconahighperf 1’ –define ‘mysqlversion 5.0.68’
for HighPerf release
The “binary” versions should work on Debian, right? (just noting Kevin’s question….the “binary” versions will run on most linux versions — just like the MySQL binaries, right?)
Also, what about Solaris? It’s not really clear what the binaries are for….
Preved Vadim,
thanks for prompt reply, it helped!
However, either source code or SRPM can not compile. It says:
Making all in sql
make[2]: Entering directory
/usr/src/redhat/BUILD/mysql-5.0.68/sql'echo “sql_yacc_H” | sed \d --debug --verbose sql_yacc.yy
make[2]: d: Command not found
make gen_lex_hash
make[2]: [sql_yacc.cc] Error 127 (ignored)
if test -f y.tab.h; then \
to=
-e ‘y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/’ \
-e ‘s/[^ABCDEFGHIJKLMNOPQRSTUVWXYZ]/_/g’
; \/usr/src/redhat/BUILD/mysql-5.0.68/sql’sed -e "/^#/!b" -e "s/Y_TAB_H/$to/g" -e "s|y\.tab\.h|sql_yacc.h|" \
y.tab.h >sql_yacc.ht; \
rm -f y.tab.h; \
if cmp -s sql_yacc.ht sql_yacc.h; then \
rm -f sql_yacc.ht ;\
else \
mv sql_yacc.ht sql_yacc.h; \
fi; \
fi
/usr/bin/perl patch_info.h.pl > patch_info.h
if test -f y.output; then \
mv y.output sql_yacc.output; \
fi
sed '/^#/ s|y\.tab\.c|sql_yacc.cc|' y.tab.c >sql_yacc.cct && mv sql_yacc.cct sql_yacc.cc
sed: can't read y.tab.c: No such file or directory
make[2]: *** [sql_yacc.cc] Error 2
make[2]: *** Waiting for unfinished jobs....
make[3]: Entering directory
if gcc -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME=”\”/\”” -DDATADIR=”\”/var/lib/mysql\”” -DSHAREDIR=”\”/usr/share/mysql\”” -DHAVE_CONFIG_H -I. -I. -I../include -I../zlib -I../innobase/include -I../innobase/include -I../include -I../include -I../regex -I. -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -MT gen_lex_hash.o -MD -MP -MF “.deps/gen_lex_hash.Tpo” -c -o gen_lex_hash.o gen_lex_hash.cc; \
then mv -f “.deps/gen_lex_hash.Tpo” “.deps/gen_lex_hash.Po”; else rm -f “.deps/gen_lex_hash.Tpo”; exit 1; fi
-mcpu=' is deprecated. Use-mtune=’ or ‘-march=’ instead./bin/sh ../libtool –preserve-dup-deps –tag=CXX –mode=link gcc -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -rdynamic -o gen_lex_hash -static gen_lex_hash.o ../myisam/libmyisam.a ../myisammrg/libmyisammrg.a ../heap/libheap.a ../vio/libvio.a ../mysys/libmysys.a ../dbug/libdbug.a ../regex/libregex.a ../strings/libmystrings.a ../zlib/libzlt.la -lpthread -lcrypt -lnsl -lm -lpthread
mkdir .libs
gcc -g -DDBUG_ON -DSAFE_MUTEX -O2 -g -march=i386 -mcpu=i686 -fno-implicit-templates -fno-exceptions -fno-rtti -rdynamic -o gen_lex_hash gen_lex_hash.o ../myisam/libmyisam.a ../myisammrg/libmyisammrg.a ../heap/libheap.a ../vio/libvio.a ../mysys/libmysys.a ../dbug/libdbug.a ../regex/libregex.a ../strings/libmystrings.a ../zlib/.libs/libzlt.a -lpthread -lcrypt -lnsl -lm -lpthread
make[3]: Leaving directory
/usr/src/redhat/BUILD/mysql-5.0.68/sql'/usr/src/redhat/BUILD/mysql-5.0.68/sql’./gen_lex_hash > lex_hash.h-t
/bin/mv lex_hash.h-t lex_hash.h
make[2]: Leaving directory
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/src/redhat/BUILD/mysql-5.0.68′
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.97483 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.97483 (%build)
Hope you could somehow help?
Cheers, Askar
Askar,
You probably need to install bison and/or flex
Sheeri,
“Binary” should work on Debian, though we did not test.
It is Linux glibc2.3 binary, we will fix description, thank you.
There are no Solaris binaries and no plans for, unless there are requests for.
Hi,
Have you compared the, “split_buf_pool_mutex” patch with Google’s smpfix patch? It sounds like the two patches are incompatible. How do they compare in stability and performance?
Sid,
It is two different patches and they solve different problems.
We are going to publish our improved version of “smpfix”, which is compatible with our patches.
i have tried rpmbuild -ba /usr/src/redhat/SPECS/mysql-percona.rhel5.spec –define ‘percona 1′ –define ‘mysqlversion 5.0.68′, but there’s an error:
-define: unknown option
Please help. Thanks