What is Apache Kafka? How is it similar to the databases you know and love, and how it's not.
Apache Kafka has become very popular in the last few years. Kafka is a distributed pub/sub server for passing data in real-time. It's fault-tolerant, scalable, and extremely fast. In this talk I will discuss Kafka's core design, how it shares core architectural features of most modern databases, and how it can speed up certain workloads by amazing amounts. I will go into detail about when to best deploy Kafka and where not to, and how to use it in conjunction with your existing database architecture to get maximum performance. This is a technical discussion with code snippets and real-world examples. You will leave this session with a better understanding of where Kafka makes sense, and where it doesn't. You will know what it's good at and what it's not. You might also leave stoked to try something new!
Kenny has decades of experience with various database platforms behind some of the busiest companies in the world. He has had roles as Founder, Architect, Director, Manager, Developer, and DBA. He was a key member of the early teams that scaled Paypal and then eBay on Oracle. He ran one of the busiest PostgreSQL installations in the world at Hi5 and was an early adopter of MongoDB using it for various large projects at Shutterfly. He is an active member in the PostgreSQL community and scaled Hi5 from just a few servers to dozens running multi-terabye workloads on SSD and SAN backends. He has contributed to the early versions of pg_reorg, and wrote the pgstat2 utility as well as other tools and performance techniques. He’s been blogging about databases including PostgreSQL for years. He has been an active MongoDB community member, speaker, MongoDB evangelist, and now Mongo Master. In 2011 he formed the MongoDB as a Service provider ObjectRocket with colleagues from eBaY. ObjectRocket was acquired by Rackspace in 2012. Currently, Kenny is a Founder at Eventador.io, a streaming data service based on Apache Kafka. He is focused on building innovative data services to power the next generation of applications that must aggregate, mutate, filter, and join data in real time.