MesosCon NA 2016 has ended
Back To Schedule
Thursday, June 2 • 9:15am - 9:45am
Keynote: Spark 2.0 - Matei Zaharia, Apache Spark Creator and CTO of Databricks

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Apache Spark is one of the most widely used systems for big data processing, and it has been integrated with Mesos from day one -- in fact, Spark started as an example framework running on the Mesos cluster manager. In this talk, I'll discuss where Spark fits in the data processing ecosystem, and some exciting additions coming out soon in Spark 2.0. In particular, unlike Apache Hadoop, Spark was designed to be a computing engine agnostic of storage systems, so that it can connect to a variety of systems beyond HDFS. This choice proved useful because most organizations use a mix of storage systems and want to run Spark on elastic infrastructure such as Mesos. Spark 2.0 will go further in this direction through support for new data sources, high-level APIs that can push key parts of the computation into the storage systems, and a new Structured Streaming API that provides similar optimizations for stream processing.

avatar for Matei Zaharia

Matei Zaharia

Keynote: Spark 2.0 - Matei Zaharia, Apache Spark Creator and CTO, Databricks
Matei Zaharia is an assistant professor of computer science at MIT and CTO of Databricks, the company commercializing Apache Spark. He started the Spark project when he was a graduate student at UC Berkeley, and continues to serve as Spark’s vice president at Apache. Matei has also... Read More →

Thursday June 2, 2016 9:15am - 9:45am PDT
Grand Mesa A-F