Over a year ago I wrote an article (multiple actually) about Software Defined Storage, VSAs and different types of solutions and how flash impacts the world. One of the articles contained a diagram and I would like to pull that up for this article. The diagram below is what I used to explain how I see a potential software defined storage solution. Of course I am severely biased as a VMware employee, and I fully understand there are various scenarios here.
As I explained the type of storage connected to this layer could be anything DAS/NFS/iSCSI/Block who cares… The key thing here is that there is a platform sitting in between your storage devices and your workloads. All your storage resources would be aggregated in to a large pool and the layer should sort things out for you based on the policies defined for the workloads running there. Now I drew this layer coupled with the “hypervisor”, but thats just because that is the world I live in.
Looking back at this article and looking at the state of the industry today, a couple of things stood out. First and foremost, the term “Software Defined Storage” has been abused by everyone and doesn’t mean much to me personally anymore. If someone says during a bloggers briefing “we have a software defined storage solution” I typically will ask them to define it, or explain what it means to them. Anyway, why did I show that diagram, well mainly because I realised over the last couple of weeks that a couple of companies/products are heading down this path.
If you look at the diagram and for instance think about VMware’s own Virtual SAN product than you can see what would be possible. I would even argue that technically a lot of it would be possible today, however the product is also lacking in some of these spaces (data services) but I expect this to be a matter of time. Virtual SAN sits right in the middle of the hypervisor, the API and Policy Engine is provided by the vSphere layer, it has its own caching service… For now it isn’t supported to connect SAN storage, but if I want to I could even today simply by tagging “LUNs” as local disks.
Another product which comes to mind when looking at the diagram is Pernix Data’s FVP. Pernix managed to build a framework that sits in the hypervisor, in the data path of the VMs. They provide a highly resilient caching layer, and will be able do both flash as well as memory caching in the near future. They support different types of storage connected with the upcoming release… If you ask me, they should be in the right position to slap additional data services like deduplication / compression / encryption / replication on top of it. I am just speculating here, and I don’t know the PernixData roadmap so who knows…
Something completely different is EMC’s ViPR (read Chad’s excellent post on ViPR) and although they may not entirely fit the picture I drew today they are aiming to be that layer in between you and your storage devices and abstract it all for you and allow for a single API to ease automation and do this “end to end” including the storage networks in between. If they would extend this to allow for certain data services to sit in a different layer then they would pretty much be there.
Last but not least Atlantis USX. Although Atlantis is a virtual appliance and as such a different implementation than Virtual San and FVP, they did manage to build a platform that basically does everything I mentioned in my original article. One thing it doesn’t directly solve is the management of the physical storage devices, but today neither does FVP or Virtual SAN (well to a certain extend VSAN does…) But I am confident that this will change when Virtual Volumes is introduced as Atlantis should be able to leverage Virtual Volumes for those purposes.
Some may say, well what about VMware’s Virsto? Indeed, Virsto would also fit the picture but the end of availability was announced not too long ago. However, it has been hinted at multiple times that Virsto technology will be integrated in to other products over time.
Although by now “Software Defined Storage” is seen as a marketing bingo buzzword the world of storage is definitely changing. The question now is I guess, are you ready to change as well?
One question I have here specifically related to VMware is about VAAI. How do some of these solutions (particularly VSAN) use VAAI, if at all? And if not, will it in the future? How does it affect performance for these kinds of storage abstraction concepts?
VSAN does not leverage VAAI… It is a different kind of storage system, different type of solutions for cloning etc are coming.
” The question now is I guess, are you ready to change as well?” And that’s the big sticking point what I’m seeing. Traditional Storage Admins are not understanding this move to Policy Based MGMT and some even refuse to accept this change. Policies, Automation are operational changes which are a harder sell, one, because IT Mgmt equates operational efficiencies to cutting staff, IT admins, the same, they are asking themselves am I putting myself at risk by promoting these technologies? To answer that question, I believe that SDDC requires a complete change away from the norm, today’s ways of operational IT both from an MGMT Operational Perspective and at the core skillset at the Engineer/Admin levels. Engineers need to become horizontal, and embrace much more than Infrastructure if they want to stay relevant. Instead of focusing on the traditional Compute/Storage/Network/Virtualization, there needs to be an acceptance of at the very least an understanding of the Application tiers, AND adding additional skillsets around automation and optimization and of course Big Data and Cloud.
While I know your article was focused on SDS, your question is a much bigger one and applies to the Data Center as a whole.
Would DataCore’s SANsymphony also be a product that would fit? Perhaps I misunderstand some of the requirements and SANsymphony is something you have already discounted, but I’d be interested to know either way.
Speaking as someone who used to use Datacore with VMware I had some fairly terrible support experiences with them pointing the finger at VMware, and refusing to take ownership of issues causing host lock ups across multiple customers.
I don’t know their product well enough to comment on it or include them in this post. Never even been briefed on what they do…
A Complimentary Research Report “Software Must Enrich Your Brand” by John McCarthy, a VP & Principal Analyst at Forrester Research, Inc http://ow.ly/x0A3X
I am curious about the exclusion of Maxta from this post… Maxta provides a software-defined solution that runs on any hypervisor and includes a full range of data and capacity optimization services. For those that are ready to make the change, there is a way today to eliminate the need for NAS and SAN devices.
There are many more vendors which were not included… These were just the ones that stood out to me, and the ones who briefed me over the last year(s).