MesosCon Europe kicks off October 8 and we’re proud to be part of the news and new technology announced at the show. In collaboration with Mesosphere, creators of the Datacenter Operating System (DCOS), we’re announcing two new open source projects for external storage volume support within Apache Mesos architectures that will be available as common services in the Mesosphere DCOS. Mesos has been getting increased traction in the “DevOps” community as an open source data center management platform that, among other things, supports container technology such as Docker.
DevOps is a broad term used in the tech industry today. Chances are, if you’ve read about DevOps, or just followed tech news, you’ve encountered the concept of containers. Containers are a technology that encapsulates software with everything it needs to run (code, runtime, system tools, system libraries and more). Since all the essentials are “contained,” the software will run the same no matter what platform it is used on. Containers tend to lean into a large focus area for customers of EMC’s Emerging Technologies Division, where infrastructure is frictionless, portable, and run as software.
The EMC {code} team, an open source and developer advocate program at EMC, has been working closely with the operational aspects of container technology over the past year to address one of the biggest challenges in the container space – enabling containers to access external or networked storage.
We saw how containers and platforms like Docker make software portable and easy, but saw that there were manual steps required to make sure that the “containerized” software could get the data it needed. In addition, we also recognized this need by customers looking to pool their data center resources with and without containers in new developer-friendly ways through technologies like Mesosphere. Focused on these opportunities we rolled up our sleeves, and created Project REX-Ray.
REX-Ray is a distributed toolset to manage storage from multiple platforms. REX-Ray locally advertises consistent methods to create, remove, map, and copy volumes abstract of what storage platform is serving the operating system.
To this date, the use case for REX-Ray and others has largely existed solely alongside the Docker engine.
Tying all of this together, let’s go into a little more detail. In Mesosphere today, frameworks and applications that consume resources typically leverage direct attached storage. Enabling external volume support will extend a Mesos agents’ ability to leverage external or network attached storage for tasks. This capability will extend DCOS’ capabilities by:
- Opening the storage platform eco-system to all of Mesos
- Reducing management complexity of external storage
- Allowing consumers of applications to specify new or existing volumes with their jobs
The EMC and Mesosphere collaboration has delivered two open source projects:
- mesos-module-dvdi – Docker Volume Driver Interface Isolator module, which resides with the Mesos agent.
- dvdcli – Docker Volume Driver CLI, works with the Isolator module and exposes support for existing Docker Volume Driver and Plugin eco-system
These two projects, working together, enable agents processing tasks to orchestrate storage for that task. Under the covers they are enabled by REX-Ray and other Docker Volume Drivers, which provide the needed orchestration abstract of container management software.
Combining existing Mesos frameworks with external volume support is a large step forward for Mesosphere, the Mesos developer community, and EMC customers. EMC is excited to be part of what’s next.
