Mesos and its distributed processing frameworks offer an easy way to migrate computation tasks to a machine-agnostic environment. To use the capabilities of Mesos to execute distributed tasks effectively a unified monitoring system is required. The monitoring system has to address the integration of disparate logging and metric facilities resulting from the heterogeneous tasks that can be executed on Mesos.
In this talk, we'll look at a framework that offers a way to ingest log, metric, and other data, and send it upstream. We'll go into detail about how the framework uses the dynamic Mesos state available on each slave to aggregate logs and metric data in a way that's transparent to running applications. It also provides a plugin-based component that can automatically ingest slave data such as detailed Mesos task resource usage statistics. We’ll also discuss the performance characteristics.
