When I started writing this article I knew people were going to say that I am biased considering I work for VMware (EMC owns a part of VMware), but so be it. It is not like that has ever stopped me from posting anything about potential competitors so it also will not stop me now either. After seeing all the heated debates on twitter between the various storage vendors I figured it wouldn’t hurt to try to provide my perspective. I am looking at this from a VMware Infrastructure point of view and with my customer hat on. Considering I have huge interest in Software Defined Storage solutions this should be my cup of tea. So here you go, my take on EMC ViPR. Note that I did not actually played with the product yet (like most people providing public feedback), so this is purely about the concept of ViPR.
First of all, when I wrote about Software Defined Storage one the key requirements I mentioned was the ability to leverage existing legacy storage infrastructures… Primary reason for this is the fact I don’t expect customers to deprecate their legacy storage all at once, if they will at all. Keep that in mind when reading the rest of the article.
Let me summarize shortly what EMC introduced last week. EMC introduced a brand new product call ViPR. ViPR is a Software Defined Storage product; at least this is how EMC labels it. Those who read my articles on SDS know the “abstract / pool / automate” motto by now, and that is indeed what ViPR can offer:
- It allows you to abstract the control path from the actual underlying hardware, enabling management of different storage devices through a common interface
- It enables grouping of different types storage in to a single virtual storage pool. Based on policies/profiles the right type of storage can be consumed
- It offers a single API for managing various devices; in other words a lower barier to automate. On top of that, when it comes to integration it for instance allows you to use a single “VASA” (vSphere APIs for Storage Awareness) provider instead of the many needed in a multi-vendor environment
So what does that look like?
What surprised me is that ViPR not only works with EMC arrays of all kinds but will also work for 3rd party storage solutions. For now NetApp support has been announced but I can see that being extended, and I now EMC is aiming to. You can also manage your fabric using ViPR, do note that this is currently limited to just a couple of vendors but how cool is that? When I did vSphere implementations the one thing I never liked doing was setting up the FC zones, ViPR makes that a lot easier and I can also see how this will be very useful in environments where workloads move around clusters. (Chad has a great article with awesome demos here) So what does this all mean? Let me give an example from a VMware point of view:
Your infrastructure has 3 different storage systems. Each of these systems have various data services and different storage tiers. Now when you need to add new data stores or introduce a new storage system without ViPR it would mean you will need to add new VASA providers, create LUNs, present these, potentially label these, see how automation works as typically API implementation differ etc. Yes a lot of work, but what if you had a system sitting in between you and your physical systems who takes some of these burdens on? That is indeed where ViPR comes in to play. Single VASA provider on vSphere, single API, single UI and self-service.
Now what is all the drama about then I can hear some of you think as it sounds pretty compelling. To be honest, I don’t know. Maybe it was the messaging used by EMC, or maybe the competition in the Software Defined space thought the world was crowded enough already? Maybe it is just the way of the storage industry today; considering all the heated debates witnessed over the last couple of years that is a perfectly viable option. Or maybe the problem is that ViPR enables a Software Defined Storage strategy without necessarily introducing new storage. Meaning that where some pitch a full new stack, in this case the current solution is used and a man-in-the-middle solution is introduced.
Don’t get me wrong, I am not saying that ViPR is THE solution for everyone. But it definitely bridges a gap and enables you to realise your SDS strategy. (Yes I know, there are other vendors who offer something similar.) ViPR can help those who have an existing storage solution to: abstract / pool / automate. Yes indeed, not everyone can afford it to swap out their full storage infrastructure for a new so-called Software Defined Storage device and that is where ViPR will come in handy. On top of that, some of you have, and probably always will, a multi-vendor strategy… again this is where ViPR can help simply your operations. The nice thing is that ViPR is an open platform, according to Chad source code and examples of all critical elements will be published so that anyone can ensure their storage system works with ViPR.
I would like to see ViPR integrate with host-local-caching solutions, it would be nice to be able to accelerate specific datastores (read caching / write back / write through) using a single interface / policy. Meaning as part of the policy ViPR surfaces to vCenter. Same applies to host side replication solutions by the way. I would also be interested in seeing how ViPR will integrate with solutions like Virtual Volumes (VVOLs) when it is released… but I guess time will tell.
I am about to start playing with ViPR in my lab so this is all based on what I have read and heard about ViPR (I like this series by Greg Schultz on ViPR). My understanding, and opinion, might change over time and if so I will be the first to admit and edit this article accordingly.
I wonder how those of you who are on the customer side look at ViPR, and I want to invite you to leave a comment.
Rob Bergin says
As a former customer – I would have to say – its all a little overwhelming and I mean the push from hardware-defined storage to software-defined storage.
And was it ever purely “hardware defined storage” – maybe when it was direct-attached disk?
Or maybe it was just the fact that many IT shops had only one big piece storage (its a good-sized capital expense) – and now the proliferation and low-cost of computers and servers is invading storage.
But even then the RAID Controller had software on it, RAM that could be upgraded for performance, etc.
I think sometimes that software-defined storage can be a little cloud-washing as non-cloudy things like SAN and NAS try and stay sexy with the public and hybrid cloud computing sexiness.
So maybe Software-Defined Storage is a way to bring that sexy back – put a layer or layers (ala hypervisor) between the unsexy hardware, hard drives and come up with tiering, pooling, automation, performance?
I do give a shoutout to the gang at DataCore, Starwind and others who have stood early on the frontlines with software based storage way before the big vendors felt it was relevant.
Shane Williford says
Nice write-up Duncan. I’m still hazy on ViPR to be honest. Maybe I’m making it more complicated than it is. Chad’s article doesn’t really help as it is so complex, though the demos help. 🙂
But, from your article, Chad’s article, etc, one thing I’d like to see is a solution even more seamless…i.e. such that tiering is even taken out of the equation. From my readings as of late, 2 ‘solutions’ that I know handle that seemingly are FPV (PernixData), and a non-SDS solution (Nimble).
All that to say, ViPR is a good start, but still has some maturing to do (all vendor solutions do on some level).
Brandon Riley says
I think you hit it on the head Duncan. I think that early players in the SDS space have been able to define what exactly it is, and since ViPR is NOT that, it’s catching people off-guard. It also is allowing the early players, and adopters to take shots at EMC b/c it doesn’t fit the narrative of the existing products calling themselves software defined storage.
As with anything that comes out against an existing narrative, it’s going to be misunderstood, and even ridiculed. If EMC is able to use their massive marketing machine to deliver a message that people can understand, they can overcome this. If they react with disdain, and attack the other SDS players already in the space, it will only perpetuate what’s happening now.
Having said all that, it’s still very early.
greg schulz says
Duncan great post and perspectives, dont worry about those who will throw mud at you. Chances are that in the future when you do a piece about somebody elses announcements, some of the current naysayers will be cheerleaders, and some of the current cheerleaders will be naysayers. Thanks for providing your perspectives good, bad or indifferent, thats the essential of independent thinking and speaking.
IMHO you mentioned something very important about ViPR, it is different from what some have seen, read or heard about thus might not be known or lacking in something to compare to. Thus there are many apple to oranges or as said in your country apples to pears comparisons or extrapolations being made. Likewise Im seeing and hearing some make or describe or think it is more complicated than it is. Granted there is a lot there or many pieces, however the basic concepts are rather straight forward, some of which have been done before directly or in variations.
Part of what has some people confused about ViPR is the out of band or fast path approach vs. what others have been doing. Another confusion point is the inclusion of block, file and object access on the north bound side, not to mention discussions of the southbound. These ideas and concepts may be new to some and thus confusing or perhaps even fear of the unknown for others not knowing what it means for them or their preferred product or protocol.
Look forward to hearing and learning more about ViPR as well as what we may see from some others as things continue to evolve.
Also thanks for the mention, appreciate.
I played with ViPR at EMC World… my initial thought was… “isn’t this what SRM (Storage Resource Management… not Site Recovery Manager) was promising half a decade (and more) ago?
That thought stayed with me for a long time.
But my thinking now is that perhaps this is a more viable option with VASA and open APIs. 5 years ago every vendor promised the ability of their SRM tool to manage 3rd party arrays… but we all know it was more promise than delivery. SMI-S never got the adoption that it promised… I could barely report on EVAs with ECC much less manage them
I’m hoping that ViPR is a delivery on the promise… but I don’t see myself being an early adopter of the software… I’m concerned about the strategy from EMC. ECC was going away and to be replaced by Prosphere… but if you have ViPR, do you still need Prosphere? What kind of revenue with ViPR need to generate to replace the revenue stream built up by ECC and Prosphere?
Just thoughts at this point.
greg schulz says
Dave I saw some of those same demos at EMCworld and can see how based on which ones were seen, or where emphasis was placed could result in confusion or perception of being another SRM (System/Storage Resource Management) or perhaps even an SRA (System/Storage Resource Analysis) tool. Granted there are the reporting tools, metrics, insight awareness features, however there is a lot more.
Likewise depending on which demo or pitch or preso or discussion, it would be and has been easy for some to jump to the conclusion that ViPR is just another variation of a Datacore, Inmage, Falconstor, SVC etc. Or perhaps similarities to some gateways for block, file or object among others.
Needless to say there are a lot of different perceptions of what ViPR is or is not, many based on what is known about it vs. known about other tools, products, services or focus areas.
Interesting you mention SMIS which was popular a decade ago, now CDMI is SNIA and its supporters luv child (and a standard btw), question is how many service providers or customers are actually demanding vs. wanting it (e.g. CDMI) today.
Erwin van Londen says
I followed up on a post from Chris Evans over here (http://massstorage.blogspot.com/2013/05/vipr-frankenstorage-revisited.html). I must say it doesn’t look new to me at all. We’ve (@ HDS) been doing this for years whilst EMC declined to acknowledge the value of it and now they scream from the towers they’ve come up with something “new”. That last part is really what I’m struggling with since a: I don’t see anything “new” and b: the end-to-end overall orchestration bit is missing.
Or am I missing something??
I sat down in the ViPR lab… but ran out of time before I had to run off to another session. The parts that I saw of it certainly seemed SRM like to me.
However, as someone who currently has 7 NetApp HA pairs, and 9 other arrays, a single tool to manage them all is appealing. NetApp hasn’t provided me with a good tool to date to help manage that sprawl.
I think the fact that you point out that SMI-S was popular a decade ago validates my point… everything old is new again. 🙂
greg schulz says
Dave that would make sense if you saw the part of ViPR more focused on metrics, measurements, reporting, etc. would make it seem like an SRM/SRA tool, along with SMIS/SNMP/custom APIs or methods for getting that info.
Granted there is much more to ViPR than just the reporting capability including the abstraction for provisioning or what some might classify as orchestration/orchestration lite. On the other hand while ViPR is targeted to work very well/integrate with VMware (e.g. DRS/SDRS/VASA/VAAI/vCenter/etc) it is not just for VMware or server virtualization. Similar while it can work in a cloud service provider (CSP) or managed service provider (MSP) or hosting or similar, it is not exclusive to those, granted initially it may be positioned that way.
Is or will ViPR be everything to everybody? Probally not.
Is there more to ViPR than what some are saying or thinking or perceiving or guessing? Yes
Do I like what EMC is doing or at least saying about ViPR? So far I like what they are saying, however they need to execute, deliver, provide more proof points, clarify who can have access to the APIs (and at what touch points in ViPR), pricing, among other questions I raised in my original post Duncan mentioned (you can find the series here: http://storageioblog.com/emc-vipr-virtual-physical-object-software-defined-storage-sds/
Do I like other tools that virtualize, provide reporting and similar functionality? Yes, different tools, technologies, vendors for various needs, requirements and situations.
Have there been other attempts to do some of the things ViPR is trying to do, particular with the out of band, fast path (e.g. get out of the data path) approach? Yes, besides EMCs own Invista, there were some that existed in Brocade or Cisco switches, as well as appliances such as the now retired HP SVSP (StorAge) which IMHO HP retired too soon.
Given your large NetApp install Dave, have you talked to them under NDA as to what they have or are thinking about? If not, might be worth a conversation with them (e.g. more than what you might get from your normal sales/support folks).
Dave as you mentioned some of the things a decade or so ago that were buzz or industry adoption trends (e.g. what the industry talks about) such as SMIS, storage virtualization, SRM etc take time to go from slideware/conceptware to broad industry deployment (e.g. what customers buy and use). This is not to say that SRM, SMIS, storage virtualization were not adopted or deployed as there have been lots of implementations.
However the full market potential has yet to be fully realized for various reasons. Thus what is old can be new, what is new may be partly old, if you have seen or experience something you are more like to have Dejavu, if you have not seen or experienced something, it is more likely to be new and revolutionary, or, in some situations scary or to be skeptical of until proven otherwise.
greg schulz says
Btw I mentioned industry adoption vs. industry deployment, here is a piece I did a few years ago about is or if there a difference (IMHO there is). However if interested you can see what I mean and am reffering to.
greg schulz says
Erwin instead of simply going by what other vendors, bloggers, pundits (myself included) say it is, or it is not, or what they know or assume or are guessing, mud, fud or real, take some time to dig or look a bit deeper. You may end up back at the same point or observation or conclusion particular if that is what looking for, otoh, you might find something different than some of the conclusions or assumptions or assertions.
Thus to your question of if you are missing something, if you are looking for it (ViPR) to be what you want it to be or seen as, then you might have meet your objectives. Otoh, maybe missing a closer look to figure out how similar to what others have done in the past, vs. different from what others have not done in the past, or what others might do.
Hope that helps.
Erwin van Londen says
Well, you know my stance. Doing this storage stuff for almost 20 years the one and only appealing to me is the reduction of admin works, costs and complexity whilst in the meantime increase business value and RAS. There have been other attempts to create an overall orchestration method, IBM Tivoli had a go as well as HP OpenView, weather or not via SNIA (CDMI/SMI-s/XAM) methods. Thus far I still have to see a singular tool which is capable of doing this in a heterogenous way. I’m not blaming EMC for giving a try but insinuating this is all new is a bit too much IMHO.
Anyway, interesting to watch on the sideline. 🙂
greg schulz says
Erwin why am I not surprised, actually, I would have been disappointed in hearing anything else ;)..
Hope all is well.
Thanks for that and looking forward to your followups. I am looking into it currently but am yet to understand it exactly. What I am not finding is someone willing to say in one sentence what it is. Is it orchestration of systems in that it will setup ESXi and connect it to my block/file storage or is it abstraction allowing any device e.g. ESXi to use any storage controlled by ViPR? I don’t see it doing that when its not in the data path.
What I do understand thus far (correct me if I’m wrong) is that it collects all VASA info and creates provisions all storage on back-end as requested by front end systems/admins. It then translates what the front end device is and is capable of and provisions storage it is capable of using and then if need be sets up access to if end to end?
Still have allot to learn but what it is not from my gatherings is vVols across all storage allowing any system to access any back-end. I guess this is where the confusion of software defined happens as provided by software platforms rather than created and administered by autonomous software. like comparing NFS to Puppet.
My one question tho is, would this be an attempt to hide generations of legacy systems with an API driven tool or more a tool for large companies who need to administer multiple systems. Most shops are not going to need this for 1 or 2 SAN’s and 50 hosts that I can see.
And off topic if EMC are heading into software then does that mean being more like Dell and buying a SAN license not tied to a frame thus living with the company through all the generations of hardware?
Arun Raju says
Nice post, Duncan. Remember that the world will have its share of people who nod their head after understanding what is blogged and you will have fanboys as well, who will disagree even you prove with logic. I found this particular article on The Register, http://www.theregister.co.uk/2013/05/10/vipr_bite_back/ which shows how the other storage vendors have responded to EMC’s ViPR.
In my opinion, I see too much of Software defined stuff in today’s world. At the end of the day, abstraction and unification of technology is what I see as a Virtualization SME will make things better, no matter if its Storage, Networking or Server Virtualization.
Matt S says
The proof is in the pudding. With VIPR, the concept is on target with what is needed in environments that are getting more complex with human resources that are staying the same or shrinking. EMC is addressing the complexity created by their “one size does not fit all” philosophy. I’ve been burnt too many times with EMC’s promises and long time to delivery to get excited about VIPR though. As an early adopter of ECC, I still have flashbacks of its instability, slowness, and data credibility. ECC/ARM was a bust. I question whether VIPR will be a viable SDS solution for a hetrogenous environment in a reasonable amount time.
The EMC strategy is fairly simple. They need to be in a position to compete with Amazon when time is right. Amazon turns everything into a utility model. It’s not yet viable but it will be and EMC must support their install base along with third party. When a customer goes to Amazon with “these” applications that have “this” iop profile and they need “this” amount of storage, Amazon will answer those two questions. That is, how much storage do you need and how fast does Amazon need to deliver it? It’s that simple. EMC needs to also answer that question without the complex data center management and build out of today.
isn’t this what falconstor/ipstor has been doing for years?
thing i don’t like about emc is everything is in pieces to be purchased. recoverypoint, data domain, avamar, networker, unisphere, vplex on top of vmax. it just becomes a soup of extra add ons to forever pay for instead of a unified global solution. so everytime you want to do something you basically have to make another product purchase. it’s almost like for the pricing you get JBOD and then they bang you on servicing those disks instead of incorporating a full solution in one. at least that’s my experience with emc.
Don’t waste your time, This is about 10 years out for full functionality. By then, we will have other type of arrays and solutions other storage technologies addressing these issues.
greg schulz says
@Jack good point and something that should be said or kept in mind context with most things being announced by various vendors/providers these days as common buzzword bingo trends/themes. Some will evolve faster than others, likewise some will be able to leverage early (hmm, early adopters) to get some benefit (business, technical or bragging rights) while others will wait. Look how long (over 20 years) it has taken nand flash SSD to get to where it is at today which is still far from its declared total dominance. Cloud, big data, storage virtualization, CDMI, open stack, software defined networking/storage/data centers/marketing/etc all have bright futures however will take time to evolve.