In this blog, we’ll look at how to manually build Percona Server for MySQL RPM packages.
Several customers and other people from the open source community have asked us how they could make their own Percona Server for MySQL RPM binaries from scratch.
This request is often made by companies that want to add custom patches to our release. To do this, you need to make some modifications to the percona-server.spec file in the source tree, and some preparation is necessary.
This post covers how you can make your own RPMs from GIT or source tarball so that you can build RPMs from your own modified branch, or by applying patches. In this example, we’ll build Percona Server 5.7.16-10.
Making your own RPMs is not a recommended practice, and should rarely be necessary.
We can fetch percona/percona-server from GitHub (or your own fork). As we build Percona Server 5.7.16-10, we create a new branch based on the tag of that version:
|
1 |
$ git clone https://github.com/percona/percona-server.git<br>Cloning into 'percona-server'...<br>remote: Counting objects: 1216597, done.<br>remote: Compressing objects: 100% (4/4), done.<br>remote: Total 1216597 (delta 1), reused 1 (delta 1), pack-reused 1216592<br>Receiving objects: 100% (1216597/1216597), 510.50 MiB | 5.94 MiB/s, done.<br>Resolving deltas: 100% (997331/997331), done.<br>Checking out files: 100% (28925/28925), done.<br><br>$ cd percona-server/<br><br>$ git checkout -b build-Percona-Server-5.7.16-10 Percona-Server-5.7.16-10<br>Switched to a new branch 'build-Percona-Server-5.7.16-10'<br><br>$ git submodule init<br>Submodule 'Percona-TokuBackup' (https://github.com/percona/Percona-TokuBackup.git) registered for path 'plugin/tokudb-backup-plugin/Percona-TokuBackup'<br>Submodule 'storage/rocksdb/rocksdb' (https://github.com/facebook/rocksdb.git) registered for path 'storage/rocksdb/rocksdb'<br>Submodule 'PerconaFT' (https://github.com/percona/PerconaFT.git) registered for path 'storage/tokudb/PerconaFT'<br>$ git submodule update<br>Cloning into 'plugin/tokudb-backup-plugin/Percona-TokuBackup'...<br>...<br>Submodule path 'plugin/tokudb-backup-plugin/Percona-TokuBackup': checked out '1b9bb16ad74588601d8fefe46c74cc1dac1dd1d5'<br>Cloning into 'storage/rocksdb/rocksdb'...<br>...<br>Submodule path 'storage/rocksdb/rocksdb': checked out '6a17b07ca856e573fabd6345d70787d4e481f57b'<br>Cloning into 'storage/tokudb/PerconaFT'...<br>...<br>Submodule path 'storage/tokudb/PerconaFT': checked out '0c1b53909bc62a4d9b420116ec8833c78c0c6e8e' |
An alternative way is to download the source tarball, which you can find at https://www.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.16-10/source/.
Extract the source tarball, as the RPM spec file is located there:
|
1 |
$ wget https://www.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.16-10/source/tarball/percona-server-5.7.16-10.tar.gz<br>$ tar -xzvf percona-server-5.7.16-10.tar.gz<br>$ cd percona-server-5.7.16-10 |
If you need to make any changes to the source code, you can either use your own GitHub fork or you can apply patches. If you use the former, then you can skip this section.
Why would we want to use patch files? Because you won’t need to maintain your own fork. You can just build the RPM with the Percona Server source tarball and the patch file.
If you do not want to use your own fork in GitHub, you can also create some patch files and modify the RPM spec file to include them.
|
1 |
$ diff -ru FILE-orig FILE >| ~/custom.patch |
In order for the patch to be applied during the building of the RPM, edit the ./build-ps/percona-server.spec file and add the two lines that are highlighted:
|
1 |
...<br>Source91: filter-requires.sh<br>Patch0: mysql-5.7-sharedlib-rename.patch<br>Patch1: custom.patch<br>BuildRequires: cmake >= 2.8.2<br>...<br>%prep<br>%setup -q -T -a 0 -a 10 -c -n %{src_dir}<br>pushd %{src_dir}<br>%patch0 -p1<br>%patch1 -p1<br><br>%build<br>...<br> |
Note that you have to number the patches, in this case I gave it the name patch1.
Creating New Source Tarball
If you use your own GitHub fork, or you made manual changes to the source (and you’re not using patch files), you should use that to create your own source tarball.
First, change the Percona Server version number. In this case, we are naming it 10custom to indicate this is not a standard build. You can adapt as you wish, just make sure the VERSION file looks something like this:
|
1 |
$ cat VERSION<br>MYSQL_VERSION_MAJOR=5<br>MYSQL_VERSION_MINOR=7<br>MYSQL_VERSION_PATCH=16<br>MYSQL_VERSION_EXTRA=-10custom |
Then make the source tarball:
|
1 |
$ cmake . -DDOWNLOAD_BOOST=1 -DWITH_BOOST=build-ps/boost<br>$ make dist<br>...<br>-- Source package ~/percona-server/percona-server-5.7.16-10custom.tar.gz created<br>Built target dist<br> |
Now you have the tarball in your source directory, but we won’t use it yet. We need to add some TokuDB submodules to it first. The make dist also kept the uncompressed directory, which we will use to create the tarball again when the TokuDB parts are included:
|
1 |
$ rm percona-server-5.7.16-10custom.tar.gz<br>$ cp -R storage/tokudb/PerconaFT/* <br> percona-server-5.7.16-10custom/storage/tokudb/PerconaFT/<br>$ cp -R plugin/tokudb-backup-plugin/Percona-TokuBackup/* <br> percona-server-5.7.16-10custom/plugin/tokudb-backup-plugin/Percona-TokuBackup/<br>$ tar --owner=0 --group=0 --exclude=.git <br> -czf percona-server-5.7.16-10custom.tar.gz <br> percona-server-5.7.16-10custom<br> |
Make sure the build host has at least 10GB of free space and at least 4GB of RAM, or the build will fail at some point.
To build the RPM, we need to prepare our build environment and ensure the necessary build dependencies are installed:
|
1 |
$ sudo yum install epel-release<br>$ sudo yum install git gcc gcc-c++ openssl check cmake <br> bison boost-devel asio-devel libaio-devel <br> ncurses-devel readline-devel pam-devel <br> wget perl-Env time numactl-devel rpmdevtools <br> rpm-build<br> |
Next we need to prepare our build directory structure, in this case we will install it in ~/rpmbuild:
|
1 |
$ cd ~/<br>$ rpmdev-setuptree |
We also need to download the boost source (http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz) and put it in the ~/rpmbuild/SOURCES/ directory:
|
1 |
$ cd ~/rpmbuild/SOURCES<br>$ wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz |
Copy the source tarball:
|
1 |
$ cp percona-server-5.7.16-10custom.tar.gz ~/rpmbuild/SOURCES/ |
Also copy the ./build-ps/rpm/* files:
|
1 |
$ cp ~/percona-server/build-ps/rpm/* ~/rpmbuild/SOURCES |
If you have any patch files, move them to the ~/rpmbuild/SOURCES/ directory as well:
|
1 |
$ cp ~/custom.patch ~/rpmbuild/SOURCES/ |
Then the percona-server.spec file goes into the ~/rpmbuild/SPECS directory:
|
1 |
$ cp ~/percona-server/build-ps/percona-server.spec ~/rpmbuild/SPECS/ |
|
1 |
$ cd ~/rpmbuild/SPECS/<br>$ sed -i s:@@MYSQL_VERSION@@:5.7.16:g percona-server.spec<br>$ sed -i s:@@PERCONA_VERSION@@:10custom:g percona-server.spec<br>$ sed -i s:@@REVISION@@:f31222d:g percona-server.spec<br>$ sed -i s:@@RPM_RELEASE@@:1:g percona-server.spec<br>$ sed -i s:@@BOOST_PACKAGE_NAME@@:boost_1_59_0:g percona-server.spec<br> |
Note the @@PERCONA_VERSION@@ contains our 10custom version number.
You can add your changelog information in the %changelog section.
For Percona Server, making the RPMs is a two-step process. First, we need to make the SRPMS:
|
1 |
$ cd ~/<br>$ rpmbuild -bs --define 'dist .generic' rpmbuild/SPECS/percona-server.spec<br>Wrote: /home/vagrant/rpmbuild/SRPMS/Percona-Server-57-5.7.16-10custom.1.generic.src.rpm<br> |
And then we can build the binary RPMs from the SRPMS:
|
1 |
$ rpmbuild --define 'dist .el7' --rebuild <br> rpmbuild/SRPMS/Percona-Server-57-5.7.16-10custom.1.generic.src.rpm<br>...<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-server-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-client-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-test-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-devel-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-shared-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-shared-compat-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-tokudb-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-57-debuginfo-5.7.16-10custom.1.el7.x86_64.rpm<br>Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.xmcI8e<br>+ umask 022<br>+ cd /home/vagrant/rpmbuild/BUILD<br>+ cd percona-server-5.7.16-10custom<br>+ /usr/bin/rm -rf /home/vagrant/rpmbuild/BUILDROOT/Percona-Server-57-5.7.16-10custom.1.generic.x86_64<br>+ exit 0<br>Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.UXUTmh<br>+ umask 022<br>+ cd /home/vagrant/rpmbuild/BUILD<br>+ rm -rf percona-server-5.7.16-10custom<br>+ exit 0 |
Once the build is done, you can find the RPMs in the RPMS directory:
|
1 |
$ ls -1 ~/rpmbuild/RPMS/x86_64/<br>Percona-Server-57-debuginfo-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-client-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-devel-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-server-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-shared-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-shared-compat-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-test-57-5.7.16-10custom.1.el7.x86_64.rpm<br>Percona-Server-tokudb-57-5.7.16-10custom.1.el7.x86_64.rpm |
Resources
RELATED POSTS