MesosCon NA 2016 has ended
Back To Schedule
Wednesday, June 1 • 3:00pm - 3:50pm
Managing Large SQL Database Clusters with the Apache Mesos Crate Framework - Aslan Bakirov & Christian Lutz, Crate.IO

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

The reliable and scalable access to databases with a standardized interface like SQL or an ORM remains a real challenge. Crate is an open source, highly scalable, shared-nothing distributed SQL database. It offers the scalability and performance of a modern NoSQL database with the power of SQL. Still, the nodes forming such a cluster need to be orchestrated.

With the introduction of persistent volumes in Mesos 0.23, reliability of stateful services improved. After a task failure or a Mesos agent restart, the stateful framework guarantees the availability of stateful data. As such it is a natural fit for a clustered database like Crate.

The Crate Mesos Framework, developed as an open source project, integrates the data storage layer with Mesos. It is a good fit for stateful services that run on top of Mesos. It avoids operations teams to manually install Crate on their cluster with resources, such as disk space, statically partitioned. Instead, it scales infinitely due to its shared-nothing architecture. The framework is highly fault tolerant: If a node or task fails, Mesos reschedules Crate instances and attaches them to the existing Crate cluster. The framework makes use of Mesos-DNS for data rebalance operations and service discovery.

The Crate Mesos Framework API simplifies managing the cluster. Using the framework API, system managers scale the cluster simply by changing the number of instances. Scaling out is easy: By using constraint rules, the Mesos framework enforces only one Crate instance to run on each host in our configuration. Scaling back by gracefully removing nodes from the cluster, is a challenge, though. The Crate Mesos framework API takes care of this task, taing into account a number of corner cases.

avatar for Aslan Bakirov

Aslan Bakirov

Software Engineer, Crate.IO
Aslan Bakirov is a Software Engineer and working at Crate.IO. He focused on big data research, worked with Mesosphere and recently joined the integrations team at Crate.IO. Before he became a Developer, he pursued a career as junior boxing champion.
avatar for Christian Lutz

Christian Lutz

CEO, Crate.io
Christian is a successful builder of software companies, including enterprise delivery platforms and big consumer portals. Previous to Crate he was co-founder of the Impossible Project (Polaroid Film Production), Fatfoogoo (in-game trading),  CEO of ucp & SVP Qpass Inc (Mobile content... Read More →

Wednesday June 1, 2016 3:00pm - 3:50pm PDT
Chasm Creek