To be fair, I have not had many customers complain about the fact that vSphere HA Admission Control was not Memory Tiering aware, but if you had concerns then it is good to know that this problem was resolved with the 9.1.1 release of VCF/vSphere. For those who never gave it any thought even, what was the problem?
When Memory Tiering is enabled, vSphere and vSphere HA considers the total amount of DRAM and the amount of NVMe of the Memory Tier as the total amount of available memory. So if you have 512GB of DRAM, and 512GB of NVMe for Memory Tiering, it would be considered as a single pool of memory of 1TB. Of course, this is the whole idea behind Memory Tiering, you have a hot and a cold tier and memory pages move up and down based on access patterns.
However, this only applies to memory pages of a VM (so for instance not of vSAN). On top of that, it only applies to memory pages which are not reserved and pinned to DRAM. On top of that, the memory overhead of a VM is also never considered for the NVMe tier. You can imagine that in certain scenarios, you could end up in the situation where vSphere HA expects there to be memory available, and even though there’s capacity in the memory pool, it is only in NVMe and not in DRAM. If that happens, it becomes impossible to restart a VM, as for instance the memory overhead always needs to be backed by DRAM.
As a result, starting with VCF/vSphere 9.1.1, the vSphere HA team has included logic for HA Admission Control to differentiate between DRAM and NVMe, and also between a tiered VM and a non-tiered VM. This by the way, only applies to the “Percentage based” admission control policy, and the “Failover Host” admission control policy. The slot policy does not differentiate between these!
Now when it comes to Admission Control and Memory Tiering, Admission Control will actually take the total amount of memory in to account (DRAM+NVMe tier) but it will also look at the capacity of DRAM only and consider that. When you set for instance the percentage to 25%, it will do the math for both DRAM as well as DRAM+NVMe, and new workloads will not be accepted unless they would pass both admission control calculations. This to prevent a situation where memory pages can be backed by NVMe and DRAM but for instance a reservation of the memory overhead cannot be guaranteerd by DRAM alone, as that would result in a failed restart.
There documentation of vSphere HA has been updated to reflect the latest changes, if you use Memory Tiering and have Admission Control enabled I would recommend looking at the documentation for the various examples of how it works. I would also highly recommend customers who use Memory Tiering and HA Admission Control to verify which policy has been selected, as that will also have an impact as explained above.
PS: the memory tiering aware HA Admission Control only comes in to play when DRS is enabled, and vSphere HA can call DRS for placement recommendations. It does not work when vCenter is unavailable and the SPE is used.
