Something I am personally very excited about is the fact that there is a beta coming up soon for an upcoming release of Virtual SAN. This beta is all about space efficiency and in particular will contain two new great features which I am sure all of you will appreciate testing:
- Erasure Coding
- Deduplication
My guess is that many people will get excited about dedupe, but personally I am also very excited about erasure coding. As it stands with VSAN today when you deploy a 50GB VM and have failures to tolerate defined as 1 you will need to have ~100GB of capacity available. With Erasure Coding the required capacity will be significantly lower. What you will be able to configure is a 3+1 or a 4+2 configuration, not unlike RAID-5 and RAID-6. This means that from a capacity stance you will need 1.3x the space of a given disk when 3+1 is used or 1.5x the space when 4+2 is used. Significant improvement over 2x when using FTT=1 in todays GA release. Do note of course that in order to achieve 3+1 or 4+2 you will need more hosts then you would normally need with FTT=1 as we will need to guarantee availability.
Dedupe is the second feature that you can test with the upcoming beta. I don’t think I really need to explain what it is. I think it is great we may have this functionality as part of VSAN at some point in the future. Deduplication will be applied on a “per disk group” basis. Of course the results of deduplication will vary, but with the various workloads we have tested we have seen up to 8x improvements in usable capacity. Again, this will highly depend on your use case and may end up being lower or higher.
And before I forget, there is another nice feature in the beta which is end-to-end checksums (not just on the device). This will protect you not only against driver and firmware bugs, anywhere on the stack, but also bit rot on the storage devices. And yes, it will have scrubber constantly running in the background. The goal is to protect against bit rot, network problems, software and firmware issues. The checksum will use CRC32c, which utilizes special CPU instructions thanks to Intel, for the best performance. These software checksums will complement the hardware-based checksums available today. This will be enabled by default and of course will be compatible with the current and future data services.
If you are interested in being considered for the beta (and apologies in advance that we will not be able to accommodate all requests), then you can summit your information at www.vmware.com/go/vsan6beta.
Go VSAN
Wow. Signed up. Possibility of dedupe and <2x free capacity! What a great VSAN team.
Exactly!
Duncan, is Dedup a post-processing or an in-line process? thx.
Great features! I hope the next release will provide more info in the GUI about vSAN disk capacity. If you have alot of admins deploying and extend disks not all are familiar or care about which disk system is used on the underlying system.
In a standard solution a 50GB disk requires 50GB of storage, and vmfs volumes can be filled to 95% capacity. If there is free space on the volume you can extend a disk. Also in a standard solution you might have a SAN or some tiering so spindles can be added and date moved around. In a vSAN solution you don’t want to fill you disks to 95%.
I a vSAN solution with FTT=1 a 50GB disk is 100GB, but this info is not available in the GUI. If there is 150GB left on the vSAN datastore a careless admin might add a 50GB disk, leaving only 50GB of raw capacity. What I would like to see is some sort of calculation in the GUI when you modify a disk what the final capacity will look like. Maybe also incorporate the vsphere warning\alarm thresholds in there, so I can set my own values for my vSAN volumes. Yellow/warning for 75% capacity used, 80% red/critical. Also if possible add rules to volumes, that force only thick or only thin disks.
Hope this makes sense 🙂
We are working on improving the operational aspects as well with every release Steffan. Especially the Health Check client is a big part of this. I will take your feedback and provide it directly to our engineering team to see how we can solve this / when we solve this.
add SSD storage tier (Tiering) and extend 600GB write buffer
Working on it…
Does this mean that the 2N+1 requirement would no longer be the case? So you would be able to have 3 objects and 1 witness on a 4 node cluster?
Yes, N+M. But this still means you can “only” tolerate 1 host failure as you have 1 parity block.
Duncan,
Kudos to the VSAN for working on data deduplication. The storage inefficiencies of shared-nothing architectures can significantly raise the total cost of these solutions.
Can you share the dedupe block size in VSAN?
— cheers,
v
Sorry, as you can understand I can’t share implementation details when the product hasn’t been released or is even in beta yet.
http://myvirtualcloud.net/?p=7312
Duncan, are you guys considering in memory optimization to accelerate workloads?
We already have CBRC for that for certain use cases. Considering the performance that VSAN provides today, we have not had any requests so far. depending of course on the hardware used, we are talking tens of thousands of iops with sub milisecond latency.
For what use case would you like to see this? And could CBRC for traditional Server Virt be something that would work for you?
The use case I can think of could be write intensive workloads; if the ACK comes from memory on two nodes and with proper UPS planning the write latency can be reduced even for those workloads.
Another one could be inline dedup before hitting the SSD preserving longevity.
These ideas are not my own but come from another vendor working in the same space that makes performance the go to market reason; I think these features could be beneficial for financial institutions and heavy OLTP as well.
What do you think about it?
“with proper UPS planning”, I am not sure we can make that assumption ever to be honest. I would always prefer to store it on persistent media, especially now that we also support NVMe and Diable Ultra DIMM which has ultra low latency that should not be a concern any longer.
When it comes to performance, I would argue we already have a very strong story and there aren’t many platforms that come close to what we can offer. Of course there are ways to optimize this, but I would prefer to do this without sacrificing availability and reliability.
NVMe and Diable Ultra DIMM are great announcements but the price point is still very high while common memory is commodity and should be used if possible.
I understand that UPS are a component outside of your stack and that you have no control over it so you don’t want to put yourself in a position where you can’t guarantee data won’t get corrupted if some event occurs.
Wow. We’re considering going vsan instead of buying another san. These features make it the holy grail if the price stays the same. I applied for the beta, so vmware, if you’re listening I have 3 hosts ready to test it out.
Support for media with 4k block sizes please!
And support for 10:1 ratio with larger faster PCIe based SSDs (i.e. 10x 2TB spinning + 1x 2TB FusionIO) would help.
Hi Duncan, I’ve heard rumours of these only being available with All Flash arrays, can you shed any light?