Percona Live: Data Performance Conference 2016 Logo

April 18-21, 2016

Santa Clara, California

/events @ Box: Using HBase as a Message Queue

/events @ Box: Using HBase as a Message Queue

 19 April 02:20 PM - 3:10 PM @ Ballroom D
Experience level: 
Intermediate
Duration: 
25 minutes conference
Tracks:
Architecture/Design
Case Stories
Topics:
Hadoop
Replication

Description

Box's /events API powers our desktop sync experience and provides users with a realtime, guaranteed-delivery event stream reflecting all of the activity happening inside their accounts. To support this functionality, we chose to leverage HBase to store and serve a separate message queue for each of our 30+ million users. This talk will cover how we implemented queue semantics on top of HBase, how we were able to replicate our queues between clusters to enable transparent client failover in the event of a complete HBase cluster outage, and why we chose to build a queuing system on top of HBase rather than leveraging a pre-existing system.

Speakers

David MacKenzie

Staff Software Engineer, Box

Biography:

David is a Staff Software Engineer at Box, where he's spent the past four years working on the infrastructure powering the company's desktop sync experience. He's currently building out Box's new HBase-backed guaranteed-delivery messaging infrastructure. Prior to Box, David worked at a small mobile telecom company building 3G network switches.

Share this talk


Slides