MesosCon NA 2016 has ended
Back To Schedule
Thursday, June 2 • 12:00pm - 12:50pm
How We Built a Chargeback System for Twitter’s Compute Platform (Mesos/Aurora) That Incentives Higher Resource Utilization - Micheal Benedict & Jeyappragash Jeyakeerthi, Twitter

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Twitter’s compute platform manages hundreds and thousands of containers across tens and thousands of hosts. At that scale, problems we have observed are:
1. Containers are not often sized appropriately in a way to utilize underlying hardware resources efficiently
2. Lack of visibility into who owns what containers (and services)
3. Identifying and tracking the utilization of resources by individual containers to the org structure

We built a Chargeback system for Mesos that provides visibility into who owns what and how much resources they use. It also generates a "bill" based on usage data. The goal here is to incentivize users of Mesos through visibility into what they use & effectively right size their containers to use the underlying resources in a cost-effective way. In this talk, we describe how we built the chargeback service that
1. Ingests utilization and quota data from Mesos master & container stats
2. process around figuring out resources to charge on (ex, cpu cores used per day)
3. ownership mapping i.e how we built a identity service that ties ldap roles with organizational teams (takes a more project/team centric view)

Towards the end of the talk, we will share a case study of how chargeback at Twitter helped improve resource utilization (cores in this case) by over 30% and save millions in $ prior to implementing something like Chargeback.

avatar for Micheal Benedict

Micheal Benedict

Head of Engineering Productivity, Pinterest
Micheal Benedict heads the Engineering Productivity organization at Pinterest that is responsible for languages strategy, source code management, build systems & CI/CD platform. Previously, Micheal led products for the Compute Platform at Twitter. Micheal holds a master's degree in... Read More →
avatar for JJ Jeyappragash

JJ Jeyappragash

Jeyappragash previously built the team and lead the technical roadmap for Twitter's Cloud Infrastructure Management Platform. This platform helps developers manage their services and provides detailed visibility to the infrastructure and the services that use the infrastructures... Read More →

Thursday June 2, 2016 12:00pm - 12:50pm PDT
Grand Mesa F