Re: Re: The Rack Endgame: A New Storage Architecture For the Data Center

I was reading Frank Denneman’s article with regards to new datacenter architectures. This in its turn was a response to Stephen Fosket’s article about how the physical architecture of datacenter hardware should change. I recommend reading both articles as that will give a bit more background, plus they are excellent reads by itself. (gotta love these blogging debates) Lets start with an out take of both articles which summarizes blog posts for those who don’t want to read the full article.

Stephen:
Top-of-rack flash and bottom-of-rack disk makes a ton of sense in a world of virtualized, distributed storage. It fits with enterprise paradigms yet delivers real architectural change that could “move the needle” in a way that no centralized shared storage system ever will. SAN and NAS aren’t going away immediately, but this new storage architecture will be an attractive next-generation direction!

If you look at what Stephen describes I think it is more or less in line with what Intel is working towards. The Intel Rack Scale Architecture aims to disaggregate traditional server components and then aggregate by type of resource backed by a super performing and optimized rack fabric. Rack fabric enabled by the new photonic architecture Intel is currently working on. This is not long term future, this is what Intel showcased last year and said to be available in 2015 / 2016.

Frank:
The hypervisor is rich with information, including a collection of tightly knit resource schedulers. It is the perfect place to introduce policy-based management engines. The hypervisor becomes a single control plane that manages both the resource as well as the demand. A single construct to automate instructions in a single language providing a correct Quality of Service model at application granularity levels. You can control resource demand and distribution from one single pane of management. No need to wait on the completion of the development cycles from each vendor.

There’s a bit in Frank’s article as well where he talks about Virtual Volumes and VAAI and how long it took for all storage vendors to adopt VAAI and how he believes that the same may apply to Virtual Volumes and Frank aims more towards the hypervisor being the aggregator instead of doing it through changes in the physical space.

So what about Frank’s arguments? Well Frank has a point with regards to VAAI adoption and the fact that some vendors took a long time to implement these. However, reality is though that Virtual Volumes is going full steam ahead. With many storage vendors demoing it at VMworld in San Francisco last week I have the distinct feeling that things will be different this time. Maybe timing is part of it, as it seems that many customers or on a crosspoint and want to optimize their datacenter operations / architecture by adopting SDDC, of which policy based storage management happens to be a big chunk.

I agree with Frank that the hypervisor is positioned perfect to be that control plane. However, in order to be that control plane for the future there needs to be a way to connect “things” to it which allows for far better scale and more flexibility. VMware, if you ask me, has done that for many parts of the datacenter but one aspect that stills needs to be overhauled for sure is storage. VAAI was a great start, but with VMFS there simply are too many constraints and it doesn’t cater for granular controls.

I feel that the datacenter will need to change on both ends in order to take that next step in the evolution to the SDDC. Intel Rack Scale architecture will allow for far greater scale and efficiency then seen ever before. But it will only be successful when the layer that sits on top has the ability to take all of these disaggregated resources, turn them in to large shared pools and allows to assign resources in a policy driven (and programmable) manner. Not just assign resources but also allow you to specify what the level of availability (HA, DR but also QoS) should be for whatever consumes those resources. Granularity is important here and of course it shouldn’t stop with availability but applies to any other (data) service that one may require.

So where does what fit in? If you look at some of the initiatives that were revealed at VMworld like Virtual Volumes, Virtual SAN and vSphere APIs for IO Filters you can see where the world is moving towards fast. You can see how vSphere is truly becoming that control plane for all resources and how it will be able to provide you end-to-end policy driven management. In order to make all of this reality the current platform will need to change. Changes that allow for more granularity /flexibility and higher scalability and that is where all these (new) initiatives come in to play. Some partners may take longer to adopt than others, especially those that require fundamental changes to the architecture of underlaying platforms (storage systems for instance), but just like with VAAI I am certain that over time this will happen as customers will drive this change by making decisions based on availability of functionality.

Exciting times ahead if you ask me.

VMware / ecosystem / industry news flash… part 2

There we go, part two of the VMware / ecosystem / industry news flash. I expected a lot of news around VMworld as traditionally is always the case. I hope the below is a good summary, these are the articles / announcements I read and found interesting. It is the Monday after VMworld and I figured I would get this out there as I will be out for most of this week to recover.

  • Maginatics: A Virtual Filer for VMware’s Virtual SAN
    Last week I mentioned the Nexenta solution for VSAN… this week Maginatics is up. They also announced it last week, but somehow it fell through the cracks so I figured I would list it this week. MSCP offers a distributed file system with global deduplication, multiple caching layers and Content Distribution Network logic build in.
  • VMware EVO:RAIL was of course all over the news, with these being my fav posts Chris Wahl, Julian Wood, Dell, Chad Sakac)
    Do I really need to comment on this one? I am hoping everyone read my blog… Also, make sure to watch the demo!
  • Infinio announced version 2.0 of their acceleration platform
    A whole bunch of announcements around the 2.0 version of Infinio Acellerator. Support for Fibre Channel, iSCSI and FCoE is probably the biggest piece of functionality added. On top of that the extension of the monitoring / reporting section is very handy for those who want to tweak based on latency / IO information you will be able to do so. There are some more features announced, make sure to read the announcement for the full details.
  • VMware joins Open Compute Project
    I was surprised about this announcement, did not know it was coming… but I am very excited. The OCP solution is interesting as it is highly optimized around efficiency / power consumption / rack units etc. I have looked at some of the configurations for Virtual SAN but the problem I saw was hardware compatibility / support. Hopefully with this announcement these constraints will be lifted soon! Definitely one I will be following with a lot of interest!
  • Nutanix announced a new round of funding: 140 million
    What more can I say than: Congratulations! Hyper-converged infrastructure is hot, and Nutanix has a compelling solution for sure. 140 million (series e) is significant, and I guess they are on their way to an IPO (rumours have been floating around for months now).

That was it for now.

VMware / ecosystem / industry news flash… part 1

I did these startup news flash newsletters for a long time and it kind of died down. Reason for it being that it felt like I missed out on updates of companies who were way beyond the stage of being a startup, which is a shame. I do feel there is a need, or there is a market for these types of blog articles. So lets give it another try but this time with a different name and a slightly different angle. In this VMware / ecosystem / industry news flash I will feature interesting (to me) news items of VMware and its ecosystem. This could range from Nexenta releasing a new version of their software to an acquisition or new round of funding. I hope you will find it useful. Besides a pointer to the originating article, I will also provide some of my thoughts, and I would like to encourage you to do the same via the comment section!

  • VMware announced the intention to acquire CloudVolumes
    I looked in to CloudVolumes about 5 months ago. I was very intrigued with what they developed and how they go about layering applications on top of virtual machines both for the server and desktop space. Especially when it comes to updating applications I was impressed how seamless and flexible CloudVolumes was. What stood out most was that they simply connect a VMDK to a VM instead of streaming data across the network… it was so simple, but yet so innovative and smart at the same time that I am not surprised VMware acquired them.
  • Ubuntu Now Available on VMware vCloud Air
    I guess there are two things here… vCloud Air formerly known as vCloud Hybrid Services, a new name. And Ubuntu being available in the catalog by default which is great. If I have to pick any Linux distribution to work with it typically is Ubuntu, and as it is one of the most used distributions out there in cloud environments this makes sense to me.
  • Nexenta Enables File Services on VMware VSAN
    I did not see this one coming to be honest and I am not sure what it will look like from an implementation perspective. It is interesting though to see the ecosystem layering services on top of VSAN and I am wondering what else is in the pipeline!
  • DataGravity out of stealth… (Steve Foskett opinion, Howard Marks thoughts, Robin Harris)
    I have read multiple articles on DataGravity and it seems they are playing in the same space as Nimble and Tintri with their hybrid storage solution. But what DataGravity adds on top of that in-sight in to the data being stored. It sounds interesting but I do wonder what the target market is as I would suspect that many who need this type of data analytics already have layers on top of their storage systems which will provide this detail for them. I will try to set some time aside at VMworld to chat with them.
  • Pernix Data announced a new round of funding (Business Insider, Venture Beat)
    35 million dollar C round of funding is significant I would say. On top the growth numbers that Pernix published are impressive, but not surprisingly if you look at what they have to offer and still have in the pipeline.
  • Atlantis announced USX 2.0 (vClouds.nl, Andrea Mauro)
    I have always been intrigued by USX. I played around with 1.0 on top of Virtual SAN and was impressed by the capabilities added and the efficiency of their data services, however at the same time I also realized that layering USX on top of Virtual SAN kind of diminished the Virtual SAN policy based management capabilities. However with USX 2.0 it appears that USX will support VVOL and as such SPBM to a full extent, and hopefully also will require a lower number of USX related virtual appliance to be deployed. Will try to explore USX 2.0 in the near future.

Startup Into: Platform9

Yesterday a startup came out of stealth which was founded by a couple of former VMware veterans. I happen to know the majority of them, and have had the pleasure to have worked with them on various things in the past. For instance Platform9‘s three of the four co-founders were all heavily involved in vCloud Director, and the fourth co-founder was VMware employee number 27… but that is not where it stops… there is much more talent on-board. But that is not what this blog is about, this blog is about Platform9, the new company that they have formed and the product they are building.

** note, I did not test the product… it is impossible to provide an analysis of what works / does not work and how they play in this space or compete with others, this is based on a conversation and a demo **

Platform9 is as they say themselves:

… the easiest way for enterprises to implement a private cloud, with intelligent, self-service provisioning of workloads onto their computing infrastructure.

  • 100% Cloud Managed: Platform9’s cloud-based model means that there is no complex management software to setup, monitor and upgrade, thus simplifying the operational experience.
  • Single Pane of Glass: Platform9 offers unified management across diverse environments – Docker, KVM and VMware vSphere – across datacenters and geographies.
  • Based on OpenStack: Platform9 customers get the best of OpenStack with 100% API compatibility.

When I met them last month, they gave a demo and showed me what they had implemented so far: Management of KVM based hypervisors using a very easy to use and slick using web-based user interface. Where the whole management solution was running in “the cloud”.

Creating a new “instance” was literally a few clicks, snapshotting / cloning it… same thing, just a couple of clicks. Now what stood out to me during the demo was the use of the word “instance” instead of “virtual machine”. So I asked them why not “virtual machine”, considering they are all VMware veterans that made more sense to me. The explanation was simple: we want to manage multiple layers. We want to manage KVM VMs, vSphere VMs but for instance also Docker containers. That is why we used a different term than we would normally use… and yes that did make sense to me. I also wondered if they would be able to mix different environments and type of instance in their UI, and the answer was yes. Docker containers, KVM VMs, vSphere VMs (etc) will also be seen in the same single pane of glass. I really like the fact that Platform9 did not limit themselves to just vSphere, or just VMs but rather is focusing on the needs of developers and providing what they require.

Similar to CloudPhysics, Platform9 is taking SaaS approach. The major benefit of course being the agility at which new features/functionality can be introduced to the outside world, or tested against a small subset of customers. Same of course applies to bug fixes / updates. No need to do that yourself, Platform9 will take care of that for you. Promising indeed.

Now there is a lot of competition in this space, as also emphasized by  Ben Kepes in his post on Platform9… But to be honest, if I look at one of the examples listed like ServiceMesh, they seem to cater for a completely different market. Platform9 is all about simplicity and managing instances, not so much about creating complex recipes etc. I agree though that there are a lot of vendors playing in this space (and as such competition), or somewhat related space, but that makes it even more interesting to see how Platform9 evolves in my opinion.

For more info, a demo, or a trial:

Platform9 will showcase its product in its booth #324 at VMware’s VMworld Conference, taking place the week of August 25th in San Francisco. The product is currently in beta with general availability planned for later this year. Platform9 currently supports KVM with upcoming support for Docker and VMware vSphere. To register for a free trial, go to: http://www.platform9.com.

CloudPhysics Storage Analytics and new round of funding

When I just woke up I saw the news was out… A new round of funding for CloudPhysics! CloudPhysics raised $15 million in a series C investment round, bringing the company’s total funding to $27.5 million! Congratulations folks, I can’t wait to see what this new injection will result in to. One of the things that CloudPhysics heavily invested in to the past 12 months has been the storage side of the house. In their SaaS based solution one of the major pillars today is Storage Analytics, along side General Health Checks and Simulations.

The Storage Analytics section is available as of today to everyone out there! It will allow you to monitor things like “datastore contention”, “unused VMs” and everything there is to know about capacity savings ranging from inside the guest to datastore level details. If you ever wondered how “big data” could be of use to you, I am sure you will understand when you start using CloudPhysics. Not just their monitoring and simulation cards are brilliant, the Card Builder is definitely one of their hidden gems. If you need to convince your management, than all you should do is show the above screenshot: savings opportunity!

Of course there is a lot more to it than I will be able to write about in this short post. In my opinion if you truly want to understand what they bring to the table, just try it out for free for 30 days here!

PS: How about this brilliant Infographic… from the people who taught you how to fight the noisy neighbour, they now show you how to defeat that bully!

**disclaimer: I am an advisor to CloudPhysics **