Yellow Bricks

by Duncan Epping

vsan esa

vSAN ESA ReadyNode configurations are more flexible than you think!

· · Leave a Comment

I had a discussion at the Dutch VMUG yesterday about the ReadyNode configurations for vSAN ESA. The discussion was about how difficult it was to select a host and customize it. It was then that I realized that most people hadn’t noticed yet that there is an easier method (or lifehack as my kids would say) when it comes to selecting your server model. How does that work? Well, let me show you!

First, let’s take a look at the vSAN ESA ReadyNode Hardware Guidance Table. The table below shows you what the node capacity is for each profile from a storage, CPU, memory, and networking perspective.

Now if you look at the table you will see that as the “profile” number goes up, so does the capacity for each of the various components. This is actually what provides you with a lot of flexibility in my opinion. If we take Dell as an example, but the same applies for most vendors on the current list, and we select “vSAN-ESA-AF2” and look at the list of options we see the following:

  • PowerEdge R650
  • PowerEdge R6515
  • PowerEdge R750
  • PowerEdge R7515

Now, if we look at “vSAN-ESA-AF8” next, which is the highest profile, we see that we only can pick 1 server model, which happens to be the PowerEdge R750. If we then look at the difference between the hosts selected for each profile a few things stand out:

  • vSAN-ESA-AF2 has an Intel Xeon Silver 4314, while vSAN-ESA-AF8 has a Platinum 8358
  • vSAN-ESA-AF2 has 512GB, while vSAN-ESA-AF8 has 1024GB
  • vSAN-ESA-AF2 a 25Gbps NIC, while vSAN-ESA-AF8 has a 100Gbps NIC
  • vSAN-ESA-AF2 has five 3.2TB NVMe devices while vSAN-ESA-AF8 has twenty-four 3.2TB devices

Now if I look at the KB article which explains what you can, and cannot change, something stands out, most of the components can be modified/customized. For instance, for CPU you can go to a higher core count and/or higher base clock speed! For memory, you can go up, same for storage devices (as long as you stay within supported limits), etc etc.

In other words, what is the difference between a vSAN-ESA-AF2 and a vSAN-ESA-AF8? Basically the expected workload, the performance, the capacity. This ultimately results in a different configuration. Nothing, at this point in time, stops you from selecting the “lowest” vSAN ReadyNode Profile and spec it as an “AF4”, “AF6” or “AF8” from a CPU stance, or from a storage/memory capacity point of view. If you want to have some more flexibility, try selecting a smaller profile, select the host type, and increase the resources/components where needed!

When you start exploring the options it may seem complex, but when you look more closely you will quickly realize that it actually isn’t that complex, and that it actually provides you with a lot of flexibility, as long as you stick to the rules and pick supported components!

What can I change about a vSAN ESA Ready Node?

· · Leave a Comment

I’ve had half a dozen people asking about this over the past weeks, it really seems more and more people are at the point of adopting vSAN ESA (Express Storage Architecture. When they look at the various vSAN ESA Ready Node configurations what stands out is that the current list is limited in terms of server models and configurations. (https://vmwa.re/vsanesahcl)

The list is being updated every week, last week for instance Supermicro popped up as a Server vendor. Of course, Dell, HPe, and Lenovo had been on the list since day 1. When you select the vendor, the ready node type, and the model you will then have the option to select a number of things, but in most cases, you seem to be limited to “Storage Device” and “Number of Storage Devices”. This however does not mean you cannot change anything. A knowledge base article has been released which describes what you can, and cannot change when it comes to these configurations! The KB article is listed on the vSAN ESA VMware Compatibility Guide list, but somehow it seems people don’t always notice the link. (Yes, I have asked the team to make the link more obvious somehow.)

Now when you look at the KB it lists what you can change, and what the rules are when it comes to making changes. For instance, you can change the CPU, but only for the same or higher core count and the same or higher base clock speed. For memory, you can increase the amount, and the same applies to storage capacity for instance. For storage it is even a bit more specific, you need to use the same make/model, so basically if the ReadyNode configuration lists a P5600 of 1.6TB, you can swap it for a P5600 of 3.2TB. However, that 3.2TB device will need to be certified for vSAN ESA as well, you can validate that using the following link: http://vmwa.re/vsanesahclc. Anyway, if you are configuring a Ready Node for ESA, make sure to check the KB so that you make supported changes!

Should I always use vSAN ESA, or can I go for vSAN OSA?

· · 4 Comments

Starting to get this question more often, should I always use vsAN ESA, or are there reasons to go for vSAN OSA? The answer is simple, whether you should use ESA or OSA can only be answered by the most commonly used phrase in consultancy: it depends. What does it depend on? Well, your requirements and your constraints.

One thing which comes up frequently is the 25Gbps requirement from a networking perspective. I’ve seen multiple people saying that they want to use ESA but their environment is not even close to saturating 10Gbps currently, so can they use ESA with 10Gbps? No, you cannot. ESA requires 25Gbps at minimum, and the bandwidth requirement fully depends on the ESA ReadyNode configuration you select. Why? Well, with ESA there’s also a certain performance expectation, which is why there’s a bandwidth requirement. The bandwidth requirement is put in place to ensure that you can use the NVMe devices to their full potential.

With both ESA and OSA you can produce impressive performance results, the big difference between ESA and OSA is the fact that ESA does this with a single type of NVMe device across the cluster, whereas OSA uses caching devices and capacity devices. ESA has also been optimized for high performance and is better at leveraging the existing host resources to achieve those higher numbers. An example of how ESA achieves that is through multi-threading for instance. What I appreciate about ESA the most is that the stack is also optimized in terms of resource usage. By moving data services to the top of the stack, data processing (compression of encryption for example) happens at the source instead of at bottom/destination. In other words, blocks are compressed by one host, and not by two or more (depending on the selected RAID level and type). Also, data is transferred over the network compressed, which saves bandwidth etc.

Back to OSA, why would you go for the Original Storage Architecture instead of ESA? Well, like I said, if you don’t have the performance requirements that dictate the use of ESA. If you want to use vSAN File Services (not supported with ESA in 8.o), HCI Mesh etc. If you want to run a hybrid configuration. If you want to use the vSAN Standard license. Plenty of reasons to still use OSA, so please don’t assume ESA is the only option. Use what you need to achieve your desired outcome, use what fits your budget, and use what will work with your constraints and requirements.

vSAN Express Storage Architecture cluster sizes supported?

· · Leave a Comment

On VMTN a question was asked about the size of the cluster being supported for vSAN Express Storage Architecture (ESA). There appears to be some misinformation out there on various blogs. Let me first state that you should rely on official documentation when it comes to support statements, and not on third party blogs. VMware has the official documentation website, and of course, there’s core.vmware.com with material produced by the various tech marketing teams. This is what I would rely on for official statements and or insights on how things work, and then of course there are articles on personal blogs by VMware folks. Anyway. back to the question, which cluster size is supported?

For vSAN ESA, VMware supports the exact same configuration when it comes to the cluster size as it supports for OSA. In other words, as small as a 2-node configuration (with a witness), as large as a 64-node configuration, and anything in between!

Now when it comes to sizing your cluster, the same applies for ESA as it does for OSA, if you want VMs to automatically rebuild after a host failure or long-term maintenance mode action, you will need to make sure you have capacity available in your cluster. That capacity comes in the form of storage capacity (flash) as well as host capacity. Basically what that means is that you need to have additional hosts available where the components can be created, and the capacity to resync the data of the impacted objects.

If you look at the diagram below, you see 6 components in the capacity leg and 7 hosts, which means that if a host fails you still have a host available to recreate that component, again, on this host you also still need to have capacity available to resync the data so that the object is compliant again when it comes to data availability.

I hope that explains first of all what is supported from a cluster size perspective, and secondly why you may want to consider adding additional hosts. This of course will depend on the requirements you have and the budget you have.

vSAN 8.0 ESA – Dude, where’s my vSAN disk group?

· · 5 Comments

Last week I was talking to a customer and he mentioned that he deployed vSAN 8.0 in his lab and he was shocked that when he wanted to define disk groups he noticed that they don’t exist anymore. Well, not in vSAN 8.0 ESA (Express Storage Architecture) that is. They do still exist in the Original Storage Architecture! The big change with vSAN 8.0 ESA is that the “bottleneck” in the previous architecture has been removed. No longer will you select a single device for caching for a particular disk group, and no longer do you designate devices purely for capacity.

With vSAN 8.0 ESA all your devices will be part of a single storage pool, and all those devices will contribute to both storage capacity as well as storage performance! The added benefit of course is the fact that writes and reads will be distributed across all devices, removing a potential choking point, and also removing a single point of failure. Why? Well with vSAN OSA when the caching device fails the whole disk group becomes unavailable. With ESA that is no longer the case as there’s no caching device!

So how does vSAN ESA provide both optimal efficiency for capacity as well as optimal performance? Well, it does this by introducing additional layers. The idea is that vSAN will provide write performance at the level of RAID-1 but space efficiency at the level of RAID-5 or RAID-6. That would be the best of both worlds. It would need to do this however while taking into consideration that we are also dealing with different types of flash devices than you normally would be with vSAN OSA. In other words, writes will also need to be optimized for the types of devices used (TLC), and it will also need to be future-proof for devices that may be supported later on (QLC).

One of the key elements in this new architecture is the introduction of the “log-structured filesystem” and the “durable log”. Let’s look at the below diagram first.

What we do with vSAN ESA is that all data is written to the log-structured file system first in the durable log. This ensures that data is persistently stored. This is what the “performance leg” provides. The performance leg literally stores the writes first. That could be 4KB blocks, or 32KB blocks, or whatever. It stores the data first, collects a full stripe write (512KB), and then writes the data to the capacity leg. Why these 2-layers? Well, the performance leg is a RAID-1 configuration, so it is optimal for write performance, while in general, the capacity leg will be RAID-5 or RAID-6, which is optimal for space efficiency. By creating this small performance leg component that holds the durable log, vSAN is capable of immediately acknowledging the writes as it is persisted in the log, and then when there’s a full stripe write it efficiently as RAID-5 or RAID-6.

Now of course, in the UI you will be able to see those new performance leg components and the capacity leg components. They are not marked as “performance” or “capacity” but they are easily recognizable. I created a quick demo that talks you through the above. If you are interested, check it out!