MesosCon NA 2016

We’ve all read the hype about the Internet of Things: a technology trend that seems endlessly about to happen, yet has been stubbornly hard to integrate with cloud computing.  In this talk I want to ask why this has been so, and how we can fix it.   

I’ll start by describing work Cornell has done over the past few years on creating a cloud platform to host “smart power grid” applications.  This involves (1) new rack-scale management solutions aimed at applications that need to run 24x7, (2) replication with ultra-fast updates for scalable real-time responsiveness,  and (3) new real-time storage solutions, to enable a new kind of big-data temporal computing.  But new models also create huge performance puzzles.  For our work, these center on how to overcome Brewer’s CAP principle.  CAP was about performance tradeoffs, and the key to conquering CAP is to leverage RDMA and NVRAM hardware, which offer amazing speedups for critical data paths.   By working from the ground up, and using RDMA and NVRAM as accelerators for data replication, storage and management solutions, we can get huge speedups compared to older styles of cloud computing, even when consistency and fault-tolerance are required.

This work is all open-source, and should be of special interest to the Apache Mesos community because container models turn out to be one of the key enablers for pulling off these tricks.  With older styles of cloud virtualization, it can be incredibly hard to offer scheduling guarantees, and technologies like RDMA and NVRAM are very hard to virtualize.  Containers eliminate both of those obstacles.