Yellow Bricks

Over the last few years, I’ve not had many discussions about component limits, but recently, these discussions have popped up more frequently, somehow. If you ask a customer what the component limit of vSAN is, some may say 9000 per host (OSA), others will say 27,000 per host (ESA), and some may know the number of the components limit per cluster even. (Documented here, and here.) However, there’s one critical component most people don’t tend to think about. For this post, I am going to focus on vSAN ESA.

As mentioned, there is a host limit, a cluster limit, but there is also a per device limit. The often made mistake is that people seem to assume that the cluster limit and the host limit are fixed limits by itself. However, there’s a dependency here. As mentioned, a device also has a limit. With vSAN ESA, a single device can hold at most 3000 data components and 3000 metadata components. This is what vSAN ESA supports today (vSAN 9.0). Now let’s focus on those data components for now. (Capacity Leg) This also means that if you have a host with 8 devices or fewer, your maximum number of components is not 27.000, but rather “the number of devices * 3000“. In other words, if you have a host with one NVMe device for vSAN ESA, the maximum number of components for that host is also 3000, if you have a host with two devices, then the max number is 6000, and so on.

So why is this worth writing about? Well, if you take the number of components per object into consideration, and you multiply that by the typical number of objects you will quickly understand why. Let’s assume you use RAID-5 with ESA in a 4+1 configuration, this would result in 5 components at least. If you have multiple disks per VM, you will easily end up with 35-40 components per VM. This means that if you look at that 3000 limit, and you divide it by, let’s take 40 components, we are talking about 75-80 VMs. Now, of course, you will have multiple hosts, so this number also multiplies per host, but hopefully this illustrates why it is important to consider this maximum number.

Then the question that remains is, why are these questions being raised now? Well, now that customers are becoming more comfortable with vSAN ESA, we are also seeing more exotic configurations. We are seeing customers deploying very large capacity devices, but with a limited number of them. Where in the past customers would use 6-8 devices per host with a capacity of 1-2TB each, more and more often I am now getting inquiries about configurations with a single 15TB NVMe device, or two 7.xTB devices. You can imagine that when you do the math, the limit of 3000 data components is far easier to reach than the 27,000 per host component limit.

So please, if you are planning for a new vSAN cluster, take these maximums into consideration. Do not just think about capacity, there’s more to take into account!