Percona Live: Open Source Database Performance Conference - Amsterdam 2016 Logo

October 3-5, 2016

Amsterdam, Netherlands

MySQL Time Machine by replicating into HBase

MySQL Time Machine by replicating into HBase

 4 October 5:20 PM - 6:10 PM @ Matterhorn 1
Experience level: 
Advanced
Duration: 
50 minutes conference
Tracks:
Big Data
MySQL
Topics:
MySQL
Hadoop
Replication

Description

At Booking.com we have complex MySQL installations, with very large tables in different servers. We’ve encountered the following question that we could not address with MySQL alone: How did some data/table look at a specific point in time? Answering this question is needed for many things, from observing trends in data changes and deriving insight, to fixing data after problems. Sadly, mistakes happen, discovery of those is not always immediate, and fixing the data later is hard if you do not know how it looked at a specific point in time. All of these become easy to solve if we have an easily accessible stream/history of data changes. However, having this is difficult because non-trivial problems need to be solved: dealing with schema changes, failover and establishing data quality guarantees. Come to this talk to learn how we solved those problems, replicating from MySQL to HBase, and implementing a Time Machine system that allows us to query data in real time at any point in the past. As a bonus, replicating MySQL data in HBase makes Hadoop integration much easier: we can now drop those Sqoop imports that were not scaling well.

Speakers

Bosko Devetak's picture

Bosko Devetak

Senior Developer, Booking.com

Biography:

Software Developer by profession, physicist by education. Has more than 10 years of experience in software development. Joined Booking.com four years ago and since then worked on data intensive projects involving Hadoop batch processing and also real time processing of high data volumes. After pushing MySQL storage and throughput to the limits, decided to test HBase at Booking.com Hackatons. After that, worked on introduction and productization of HBase into Booking.com infrastructure. During last year he was mostly focused on MySQL to HBase replication. He is the author of “MySQL Time Machine” (mysql-time-machine on github).

Rares Mirica's picture

Rares Mirica

Senior Developer, Booking.com

Biography:

Self-taught IT professional with a background in web-development, database administration and systems administration. Focused on web infrastructure and its interface with the application development process. Works at Booking.com as an infrastructure engineer focusing on adoption of new technologies into the stack.

Share this talk


Slides