MesosCon NA 2016 has ended
Back To Schedule
Wednesday, June 1 • 5:20pm - 6:10pm
Spark on Apache Mesos: The State of the Art - Tim Chen, Mesosphere

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Multi region global Data centers can be setup by using Mesos as the Kernel for all resource management. This presentation will cover one of the designs for federated mesos clusters to enable uniform data center resource orchestration across the globe. We will also compare and contrast various design approaches that has been considered for this project including some industry’s competitive models. The Model we propose is not only distributed by nature, it can also tolerate dynamic addition/deletions/modifications of masters in the federation. We will also explain how a framework can participate in such a design without having to change its core functionality.

A policy engine is integrated with this cluster federation model to control the nature of the scheduler. This federation work will not only handle the IaaS layer resource requirements but also extended to deal with PaaS layer application resources. So in the container world, Mesos can play a big role directly handling the applications which could avoid other PaaS resource schedulers by using custom frameworks. Various optimum schedulers can be pluggable to this model and health of the overall system is taken care by inter cluster communication in HA mode. Finally we will show a POC demo developed within our lab and probably the ongoing implementation details in our cloud.


Tim Chen

Distributed Systems Engineer, Mesosphere
Tim Chen is on the Mesosphere core team and focuses on containerization and big data frameworks. He is also a PMC/committer on Apache Drill and Apache Mesos, and contributed to other open source projects such as Spark, Kafka, Hive. Before joining Mesosphere, Tim past experiences includes... Read More →

Wednesday June 1, 2016 5:20pm - 6:10pm PDT
Grand Mesa ABC