Running Hadoop on Mesos at Uber
19 April 01:20 PM - 2:20 PM @ Ballroom E
50 minutes conference
Companies often have to build their infrastructure to handle the peak load of their systems while being over-provisioned during off-peak times. At Uber we are leveraging Apache Mesos, Apache Aurora, and other frameworks to make our systems more efficient, reliability and to take advantage of excess off-peak capacity for use in resource-intensive jobs like Hadoop. We will go over why we are using Apache Mesos, the pros and cons of the frameworks we are using, and the benefits and challenges of running a multi-framework environment to handle the various services and jobs we have. Additionally we will discuss the design and implementation of our solution and show how we use puppet to tie it together. The presentation will also cover operation and management of our Apache Mesos cluster to make the shifting of resources easy and the loss of hardware seamless. The target of this talk is DevOps but it should be accessible to a general audience. The goal of this presentation is to provide a general understanding of how to set up and use Apache Mesos and which frameworks are suited for which jobs.
Senior Software Engineer, Uber
Brandon has been working in the tech industry for over 15 years. Recently he has been working at Uber to leverage Apache Mesos to help improve datacenter utilization.