MySQL 8.0.17 Clone Plugin: How to Create a Slave from Scratch

MySQL 8.0.17 Clone PluginIn this post, we will discuss a new feature – the MySQL 8.0.17 clone plugin. Here I will demonstrate how easy it is to use to create the “classic” replication, building the standby replica from scratch.

The clone plugin permits cloning data locally or from a remote MySQL server instance. The cloned data is a physical snapshot of data stored in InnoDB, and this means, for example, that the data can be used to create a standby replica.

Let’s go to the hands-on and see how it works.

Installation & validation process of the MySQL 8.0.17 clone plugin

Installation is very easy and it works in the same as installing other plugins. Below is the command line to install the clone plugin:

And how to check if the clone plugin is active:

Note that these steps need to be executed on the Donor (aka master) and on the Recipient (aka slave if the clone is being used to create a replica).

After executing the installation, the plugin will be loaded automatically across restarts, so you don’t need to worry about this anymore.

Next, we will create the user with the necessary privilege on the Donor, so we can connect to the instance remotely to clone it.

As a security measure, I recommend replacing the % for the IP/hostname or network mask of the Recipient so the connections will be accepted only by the future replica server.  Now, on the Recipient server, the clone user requires the CLONE_ADMIN privilege for replacing recipient data, blocking DDL during the cloning operation and automatically restarting the server.

Next, with the plugin installed and validated, and users created on both Donor and Recipient servers, let’s proceed to the cloning process.

Cloning process

As mentioned, the cloning process can be executed locally or remotely.  Also, it supports replication, which means that the cloning operation extracts and transfers replication coordinates from the donor and applies them on the recipient. It can be used for GTID or non-GTID replication.

So, to begin the cloning process, first, let’s make sure that there’s a valid donor. This is controlled by clone_valid_donor_list parameter. As it is a dynamic parameter, you can change it while the server is running. Using the show variables command will show if the parameter has a valid donor:

In our case, we need to set it. So let’s change it:

The next step is not mandatory, but using the default log_error_verbosity the error log does not display much information about the cloning progress. So, for this example, I will adjust the verbosity to a higher level (on the Donor and the Recipient):

Now, let’s start the cloning process on the Recipient:

It is possible to observe the cloning progress in the error log of both servers. Below is the output of the Donor:

And the Recipient:

Note that the MySQL server on the Recipient will be restarted after the cloning process finishes. After this, the database is ready to be accessed and the final step is setting up the replica.

The replica process

Both binary log position (filename, offset) and GTID coordinates are extracted and transferred from the donor MySQL server instance.

The queries below can be executed on the cloned MySQL server instance to view the binary log position or the GTID of the last transaction that was applied:

With the information in hand, we need to execute the CHANGE MASTER command accordingly:

Limitations

The clone plugin has some limitations and they are described here. In my opinion, two major limitations will be faced by the community. First, it is the ability to clone only InnoDB tables. This means that  MyISAM and CSV tables stored in any schema including the sys schema will be cloned as empty tables.

The other limitation is regarding DDL, including TRUNCATE TABLE, which is not permitted during a cloning operation. Concurrent DML, however, is permitted. If a DDL is running, then the clone operation will wait for the lock:

Otherwise, if the clone operation is running, the DDL will wait for the lock:

Conclusion

Creating replicas has become much easier with the help of the MySQL 8.0.17 clone plugin. This feature can be used using SSL connections and with encrypted data as well.  At the moment of publication of this blog post, the clone plugin can be used to set up not only asynchronous replicas but provisioning Group Replication members too. Personally, I believe that in the near future this feature will also be used for Galera clusters. This is my two cents for what the future holds.

Useful Resources

Finally, you can reach us through the social networks, our forum or access our material using the links presented below:

Share this post

Comments (8)

  • Adam Mulla Reply

    I have read you article ….very informative… Thank you…

    I think Galera cluster with mariadb Right? It is not related to MySQL.

    August 1, 2019 at 12:29 pm
  • Bruno Cabral Reply

    This is incredible. Any plan to support RocksDB?

    August 1, 2019 at 12:55 pm
    • Vinicius Grippa Reply

      Hi Bruno,

      We are constantly developing Percona Server versions. Currently, the RocksDB engine is supported on Percona Server versions. Because MySQL 8.0.17(which contains the clone plugin) was released only a few days ago (10 days from the time of this post), we still don’t have the roadmap for this feature. You can check our blogs or sign the newsletter to follow our releases.

      August 1, 2019 at 1:08 pm
  • lefred Reply

    Maybe it’s time to promote MySQL InnoDB Cluster (Group Replication) with Percona Server too 😉

    August 1, 2019 at 1:15 pm
  • Awesome Reply

    A couple thoughts as we start testing it out:

    * Compression does not seem to be used during a clone which may take longer than say PXB
    * Same for multiple file copy threads, compared to xbstream
    * Master disk usage may increase in the #ib_archive directory

    Pretty exciting to see, also cool you can specify DATA DIRECTORY for remote backups

    tag: load data from master

    August 1, 2019 at 2:57 pm

Leave a Reply