Unfortunately I haven’t been able to attend too many sessions, only 2 so far. This is one I didn’t want to miss as it was all about what VMware is working on for VSAN and layers that could sit on top of VSAN. Rawlinson and Christos spoke about where VSAN is today first. Mainly discussion the use cases (monolithic apps like Exchange, SQL etc.) and the simplicity VSAN brings. After which an explanation of the VSAN object/component model was provided which was the lead in to the future.
We are in the middle of an evolution towards cloud native applications Christos said. Cloud native apps scale in a different way then traditional apps, and their requirements differ. Usually not a need for HA and DRS, and will contain this functionality within their own framework. What does this result in for the vSphere layer?
VMware vSphere Integrated Containers and VMware Photon Platform enabled these new types of applications. But how do we enable these from a storage point of view? What kind of scale will we require? Will we need different data services? Will we need to different tools, what about performance?
First project being discussed is the Performance Service which will come as part of the Health Check plugin. Providing cluster level, host level, disk group level, disk level… The Performance Service Architecture is very interesting and is not a “standard vCenter Server service”. Providing deep insights using per host traces is not possible as it would not scale. A distributed model is proposed which will enable this, but in a decentralized way. Each host can collect data, each cluster can roll this up, and this can be done for many clusters. Data is both processed and stored in a distributed fashion. The cost for a solution like this should be around 10% of 1 core on a server. Just think what a vCenter Server would look like if you had the same type of scale and cost, with a 1000 host solution could easily result in a 100 vCPU requirement, which is not realistic.
Rawlinson demoes a potential solution for this, in this scenario we are talking 1000s of hosts of which data is gathered, analyzed and presented in what appears to be an HTML-5 interface. The solution doesn’t just provides details on the environment it also allows you to mitigate these problems. Note that this is a prototype of an interface that may or may not at some point in time be released. If you like what you see though, make sure to leave a comment as I am sure that helps making this prototype reality!
Next being discussed is the potential to leverage VSAN not just for virtual machines, but also for containers, having the capabilities to store files on top of VSAN. A distributed file system for cloud native apps is now introduced. Some of the requirements for a distributed file system would be a scalable data path, clones at massive scale, multi-tenancy and multi-purpose.
VMware is also prototyping a distributed file system and have it running in their labs. It sits on top of VSAN and leverages that scalable path and uses it to store its data and metadata. Rawlinson demonstrates how he can create 2000 clones of a file in under a second across a 1000 host and runs his application. Note that this application isn’t copied to those 1000 hosts, but it is a simple mountpoint on 1000 hosts, truly distributed filesystem with extremely scalable clone and snapshot technology.
Christos wraps up, key points are that VSAN will be the enabler of future storage solutions as it provides extreme scale, with at a low resource overhead. Awesome session, great peak in to the future.
[email protected] says
Marc-Andre Labelle (@hoplawego) says
About version 2.0 of vSan. Will we be able to create more than one Vsan datastore?
John Nicholson says
Can you share what is your use case for more than one VSAN datastore on a cluster? You can have multiple disk groups in a host, and with fault zone awareness now in 6 you can isolate out hosts to survive a rack failure, etc within the same cluster. Storage policies (Failure to tolerate, striping, cache reservation) are all dynamic and granular at the VMDK level so you do not need multiple clusters to tune different policies.
I have heard some request to be able to deploy 3 tiers of disk (NL, 10K, and flash) but the general downward trend on flash pricing (~56 cents a GB drives on the HCL vs. ~25 cents for a 10K drive) combined with Dedupe for flash (expecting around 2:1) means that all flash VSAN will likely cover the “need consistent low latency random read” performance use cases. Hybrid with NL-SAS will cover the “high data skew” use cases. While 10K has a place today still, I expect it to get squeezed out over time as the road-map for flash vendors is looking very aggressive. The VIP sizing tool can capture how random your workload is (and to help design how much cache you need). It can help take the mystery out of guessing what design you need for a great outcome.
Great post, Duncan! Your readers may also find real users reviews of VMware VSAN helpful if they want to learn more about how and why they use the solution. As an example, this user writes that with VSAN, “you set the policy, and you will get the level of redundancy and resiliency that you want. Something that has been in the enterprise space for quite sometime, with some of the more expensive arrays, now you can bring it down into the commercial even the mid-market space. That’s pretty amazing.” You can read the rest of his review here: https://goo.gl/8SNp77