I could visit just one session on Tuesday so I decided I would attend a session which is on a topic that I love “Software Defined Storage”, aka vCloud Distributed Storage. This session was hosted by Christos Karamanolis and Kiran Madnani. Christos is one of the lead engineers on the project and Kiran is the product manager, both VMware employees.
Before I lay down what their session was about, note that this was a technology preview. No commitments have been made when or even IF this will ever be released… but as the demo looked so promising I figured it was worth sharing.
[Edit: Cormac Hogan published the demo, worth watching!]
Kiran started out with explaining the journey to Software Defined Storage. Awesome features he mentioned were Storage vMotion, Storage DRS, SIOC, Profile Driven Storage, vSphere Replication and more. Kieran made a statement which I can only agree with….
The world is changing, compute resource keep growing, data keeps on growing!
Data management is a challenge in today’s world. In many environments virtual machines are provisioned on tier 1 storage just to ensure requirements are met, while in many cases that is overkill. High storage costs, inconsistency and operational complexity are the direct results of that.
So if VMware had to solve this problem and create a new flexible / agile storage platform, what would the requirements be? Customers asked us: “Why don’t you virtualize local storage the same way you virtualized CPU and memory?”
These are the requirements Kiran mentioned:
- Converged platform (compute + storage using commodity hardware)
- Simple to manage just like CPU and memory
- Ability to scale on demand, in lock-step with application needs, but also in a flexible way
- Provide per-VM automated SLA management
Now Christos entered the stage. Christos being an engineer, I knew this was going to get more technical. Christos explained that he is working on what is now known as “VMware Distributed Storage” but also sometimes referred to as vSAN.
This new solution/product is a software based distributed storage solution which is built directly in the hypervisor. I guess this is also the main differentiator compared to competitors, it is not a virtual machine / appliance! Enabling it will be simple, it should be just two clicks. And it will be a property of a vSphere Cluster. When enabled a shared datastore is presented. This datastore grows as the cluster grows. This datastore is formed by using local storage (SSD / SATA / SAS) of the host.
Note that not all hosts need to be identical. Some hosts can have storage others might not. Christos demoed this during the presentation, he had a virtual machine running on a host without local storage while running on a “remote” VMware Distributed Storage datastore. This virtual machine was then vMotioned to a different host which also did not have local storage, but still had access to the VMware Distributed Storage shared datastore.
VMware Distributed Storage heavily relies on SSD for caching / performance… eventually though all writes will go to magnetic disks (SATA) eventually. For resiliency “objects” will be replicated across multiple hosts, how many hosts/disks will depend on the level of availability selected for the object.
This in my opinion one of the biggest advantage of a software defined storage solution, you can define an SLA per virtual machine. Yes you read that correctly, policies are on a virtual machine level and not on a datastore level. You need 99.99% uptime or 99.999% uptime, VMware Distributed Storage can do that for you on the same datastore. Not only can you specify availability but also IOPs (reservations and limits) or capacity. Note that VMware Distributed Storage doesn’t use a typical RAID solution, it uses JBOD (just a bunch of disks) and the software takes care of the distribution / replication of the data.
The one thing that stood out for me during this session was the demo. It was so simple, I wonder if people caught on to that. Enabling VMware Distributed Storage was only 2 or 3 clicks and that was it. Expanding the datastore was just a matter of adding a new host (or adding disks to a host for that matter).
I noted down the specs of the systems used for the demo. This is no guarantee of what will eventually be supported or part of the requirements but it gives a nice idea around what a VMware Distributed Storage environment could look like, I am guessing though you will want more local disks though.
- 7 ESX hosts, each with:
- Quad-Core Intel Xeon E5620 2.4GHz processor
- 24 GB DDR3 1333MHz memory module
- 10 Gb/s Dual-channel, 10GBase-CR, PCI-Express 2.0
- 4 ESX hosts have local storage:
- 1 x 500GB SAS 6.0Gb/s HDD
- 1 x 256GB SAS SSD
So now what would be the usecase for this? During the session 4 usecases were mentioned explicitly:
- Virtual desktops
- Scale out model, using predictive (performance etc) repeatable infrastructure blocks lowers costs and simplifies operations
- Test & Dev
- Avoids acquisition of expensive storage (lowers TCO), fast time to provision
- Big Data
- Scale out model with high bandwidth capabilities
- Disaster recovery target
- Cheap DR solution, enabled through a feature like vSphere Replication that allows to replicate to any storage platform
It is funny though, as I would have probably added cloud on top of that… and potentially just production server workloads. Kiran ended summarizing vSAN / VMware Distributed Storage as:
- Software-driven – use industry standard hardware
- Flexible – scale as needed, when needed
- Simple – ridiculously easy to manage
- Automated – per-VM policy-based management
The simplicity and the flexibility of this product is what stood out the most, I personally believe this has huge potential. This will be big!
A couple of articles I would like to refer to: