MesosCon NA 2016

Mesos currently supports two types of container runtimes: one is Mesos’ own pluggable runtime which uses OS primitives directly (a.k.a. the Mesos containerizer); the other is to delegate the entire container management to Docker engine (a.k.a. the Docker containerizer). Recently, we added container image support to Mesos containerizer, which allows it to launch off-the-shelf Docker or Appc containers without depending on Docker daemon or rkt. In this talk, we are going to discuss the motivation of why we want to unify the containerizers in Mesos and how this benefits the Mesos maintainers and our users. We will then talk about the current status of this work, and show what can be built on top of this pluggable architecture. Finally, we will share our vision on the near future for the Mesos containerizer and how it’s going to enable more innovation in the Mesos ecosystem.

Mesos will soon reach the 1.0 milestone. In addition to new features, this would mean a more stable user-facing API and stricter support/release guarantees for operators/framework developers. The aim of this talk is to apprise the operators/framework developers/users about the new API and also discuss the support/compatibility guarantees offered by Mesos going forward.

This talk is a sequel to the last year’s MesosCon Seattle talk on “Mesos HTTP API” and continues from where it left off.

This talk will cover the following specific topics:
- Discuss the newly introduced Operator APl.
- Update on the recent improvements to the Framework API.
- Update on client libraries for the new Framework API.
- Release cadence for Mesos going forward.
- Support/Compatibility guarantees for operators/framework developers e.g., backporting of patches etc.
- Master->Agent renaming in the 1.0 API.

Upgrading a running cluster can be a daunting task, but do not fear: Mesos masters and agents intelligently persist their state to allow recovery after restart, which enables upgrades with zero downtime! In this talk, we’ll cover the best practices and potential pitfalls you should be aware of before performing a live upgrade of a Mesos cluster: how to properly kill and restart Mesos master/agent processes, how and when to recompile Mesos schedulers and modules, and what to do when things go wrong.

CERN developed and is using CernVM File System (CVMFS) to distribute terabytes of software to hundreds of thousands of nodes across hundreds of datacenters. CVMFS uses a combination of lazy loading, extensive indexing, deduplication, caching and geographic distribution. This help to significantly reduce the network load and speed-up the distribution.
All these features make CVMFS a very attractive candidate for solving container-image management and distribution problem in Mesos.
In this talk we are going to present our work on integrating CVMFS client with Mesos agents as well as show how CVMFS backend can be used as a secure and scalable container image registry. We will also run a demo in which we will show how can one launch a container running a popular Linux distribution (image size is about ~300MB) by only downloading a tiny fraction of the image (~6MB).

At Adobe APIs are powering the next generation of Creative applications.

Mesos makes it very easy and fun to deploy and run Robust and Scalable Microservices in the Cloud. Today's technologies offer simple solutions to create RESTfull services while Mesos brings them to life faster.

As the number of microservices increase and the inter communication between them becomes more complicated, we soon realize we have new questions awaiting our answers: how do microservices authenticate ? how do we monitor who's using the APIs they expose ? How do we protect them from attacks ? How do we set throttling and rate limiting rules across a cluster of microservices ? How do we control which service allows public access and which one we want to keep private ? How about Mesos APIs and its frameworks ? Can they benefit from these features as well ?

Come and learn a scalable architecture to manage microservices in Mesos by integrating an API Management layer inside your Mesos clusters. This presentation will show you what an API Management layer is, what it's composed of and how it can help you expose microservices in a secure,managed and highly-available way, even in multi-Mesos cluster setups.

During this session you will also have the opportunity to learn how Adobe's API Platform solved this problem, where it is today and what it envisions do to with Mesos further.

If you're working with microservices already or you're creating new ones then this presentation is for you. Come and learn how Mesos together with an API management layer will make you a microservices hero in your organisation.

In this presentation Carlos Sanchez will share his experience running Jenkins at scale, using Docker and Apache Mesos to create one of the biggest (if not the biggest) Jenkins clusters to date.

By taking advantage of Apache Mesos, the Jenkins platform is dynamically scaled to run jobs across hundreds of Jenkins masters, on Docker containers distributed across the Mesos cluster. Jenkins slaves are dynamically created based on load, using the Jenkins Mesos and Docker plugins, running in containers distributed across multiple hosts, and isolating job execution.

This presentation will allow a better understanding of Apache Mesos and the challenges of running Docker containerized and distributed applications, particularly JVM ones, by sharing a real world use case, including good and bad decisions and how they affected the development.

Whether it is Docker or the Mesos Container running with Apache Mesos means running in a container, but do you really understand cgroups and how they work? Do you understand the difference between CPU Sets and CPU Shares? Many organizations use Java for their application development, but do you understand the consequence of what impact the cgroup constraints have on the JRE?

This talk starts with a look at the anatomy of a container including control groups (cgroups) and namespaces. Then we will take a deep understand of Java's memory management and GC characteristics and how JRE characteristics change based on core count. The session will detail specifically the difference between CPU sets and CPU shares and memory management. The session will close with a deep understanding of the consequences of running the JRE in a CPU share environment and the potential for pseudo-random behavior of running in a heterogeneous datacenter.

Mesos provides the ability to harness the resources of a compute cluster, and provides detailed insights into those compute resources. This made Mesos a great fit at Tendril for their microservices architecture that powers their Energy Services Management platform. The challenge arose as to how to best leverage Mesos to dynamically scale individual services based on workload. David Arnold will cover how Tendril created an autoscaling solution with Mesos and the Marathon framework running on Amazon Web Services.

Everyone knows how microservices and containers are revolutionizing the way we deploy applications and maintain our infrastructure. But as many teams are starting to find out, containers still have a key challenge: monitoring and troubleshooting them can be impractical, painful, and sometimes plain impossible. With the rise of microservice based architectures and container orchestration tools such as Mesos and DCOS managing difficult task has become a little bit easier - if you take the right approach.

In this demo-driven presentation, Loris Degioanni will cover the current state of the art of container and microservice monitoring, including the pros and cons of some popular approaches. Using real tools running in live environments, he will demonstrate how to effectively monitor, explore and troubleshoot highly scaled Mesos deployments.

The presentation will feature live interaction with container environments and live demos of all tools and techniques discussed. Special emphasis will be put on using the Mesos portfolio of scheduling and management tools as well as sysdig, an open source container and system troubleshooting tool developed by the presenter.

GitHub link: https://github.com/draios/sysdig

Specific topics will include:
* visualizing the physical vs logical architecture of Mesos & DCOS deployments
* understanding performance at the holistic microservice/application level for orchestrated systems
* Leveraging Mesos metadata such as Master, Slave, Marathon and labels for more intelligent troubleshooting
* extracting process and application-level performance metrics from inside containers using non-intrusive methods
* troubleshooting detailed network activity among distributed containers

Mesos is a core component of the Mantl project (Mantl.io), an open-source platform that makes it easy to deploy containerized workloads and streaming data applications.

Mantl integrates Mesos with a number of tools to provide security, high availability, service discovery, health checks, and load-balancing. It deploys to most cloud providers in minutes.

This talk will discuss the architecture and engineering of Mantl, from using Terraform to orchestrate multi-cloud deployments, Mesos-Consul for service discovery, load-balancing via Traefik, Mantl-api for launching frameworks, and Calico for Overlay networking.

Mesos is by far the best orchestration system available out there for running containers. At YP, when we started using Mesos couple of years ago, we were overwhelmed with the robustness it provided. We quickly started on-boarding some of the apps. We soon were faced with multiple request from various developers to on-board their apps onto Mesos. These apps ranged from infrastructure tools to highly critical billing systems, from traditional java to web based apps, from map reduce jobs to batch processing jobs, from one off cron jobs to build pipeline jobs. These apps were written in Java, Go Lang, Node.js, Ruby/Rails, C, Scala as well as Python. And their configuration came from git repo, key value store, pom files or some static files managed somehow. We were made to provide a unified platform that served the use-cases for all of them or at least come up with the processes that everyone can use.

Initially, we relied heavily on the out of the box features that Mesos provided. But that wasn’t enough to support these myriad range of apps. For eg: redirecting all logs to STDOUT/STDERR wasn’t enough or possible for some of these apps. Managing app secrets was very insecure and exposed via ENV variables in Marathon. Also, managing app configs for various environment was tedious. We wanted to achieve persistence and fault tolerance with block devices to run databases through Mesos. There was no easy way to troubleshoot running processes in production. There was a need to manage multiple similar frameworks within the same Mesos cluster. And the list goes on and on.

We, at YP, having been exposed to this madness. While doing so, we have picked up valuable information along the way. We may not have solved all the issues, but we have made significant strides to resolve them.

All Marathons need a runner. Introducing Pheidippides (John Dennison & James Humphrey, Activision Publishing, Inc). With Docker microservices taking the world by storm, John and James cover the challenges with the transition, approach, and their solution to this problem. Introducing a new service named Pheidippides (the apocryphal runner of the first marathon), built to help deploy, configure, and manage services within the Mesos, Marathon, and Chronos ecosystem.