Those who have configured vSphere HA have all seen that section where it asks if you want to use admission control or not. Of course if you decide you want to use it, and you should want this, then the next question that comes is which one do you want to use? I have always preferred the “Percentage Based Admission Control” policy. For some reason though there are people who think that the percentage based admission control policy rules out large VMs from being restarted or offers a lower guarantee.
The main perception that people have is that the percentages based admission control policy gives lower guarantees of virtual machines being restarted than the “host failures” admission control policy. So let break it down, and I mean BREAK IT DOWN, by using an example.
Example
- 5 hosts
- 200GB of Memory in cluster
- 20GHz of CPU in cluster
If no reservations are set:
Percentage Based will do the following:
- The Percentage Based policy will take the total amount of resources and subtract the amount of resources reserved for fail-over. If that percentage is for instance 20% than 40GB and 4GHz are subtracted. Which means 160GB and 16GHz are left.
- The reserved resources for every virtual machine that is powered on is subtracted from what the outcome of 1. was. If no reservation is set memory then memory overhead is subtracted, if the memory overhead is 200MB then 200MB is subtracted from the 160GB that was left resulting in 159,8GB being available. For CPU the default of 32MHz will be used.
- You can power-on virtual machines until the amount of available resources, according to HA Admission Control, is depleted, yes many VMs in this case.
Host Failures will do the following:
- The Host Failures policy will calculate the amount of slots. A slot is formed out of two components: memory and cpu. As no reservation is used the default for CPU is used which is 32MHz, with vSphere 5.0 and higher. For memory the largest memory overhead size is used, in this scenario there could be a variety of sizes lets say the smallest is 64MB and the largest 300MB. Now 300MB will be used for the Memory Slot size.
- Now that the slotsize is known Admission Control will look for the host with the most slots (available resources / slot size) and subtract those slots from the total amount of available slots. (If one host failure is specified). Every time a VM is started a slot is subtracted. If a VM is started with a higher memory reservation we go back to 1 and the math will need to be done again.
- You can power-on virtual machines until you are out of slots, again… many VMs.
If reservations are set:
Percentage Based will do the following:
- The Percentage Based policy will take the total amount of resources and subtract the amount of resources reserved for fail-over. If that percentage is for instance 20% than 40GB and 4GHz are subtracted. Which means 160GB and 16GHz are left.
- The reserved resources for every virtual machine that is powered on is subtracted from what the outcome of 1 was. So if 10GB of memory was reserved, then 10GB is subtracted resulting in 150GB being available.
- You can power-on virtual machines until available resources are depleted (according to HA Admission Control), but as reservations are used you are “limited” in terms of the amount of VMs you can power-on.
Host Failures will do the following:
- The Host Failures policy will calculate the amount of slots. A slot is formed out of two components: memory and cpu. As a reservation is used for memory but not for CPU the default for CPU is used which is 32MHz, with vSphere 5.0 and higher. For memory there is a 10GB reservation set. 10GB will be used for the Memory Slot size.
- Now that the slotsize is known Admission Control will look for the host with the most slots (available resources / slot size) and subtract those slots from the total amount of available slots. (If one host failure is specified). Every time a VM is started a slot is subtracted, yes that is a 10GB memory slot, even if it has for instance a 2GB reservation. If a VM is started with a higher memory reservation we go back to 1 and the math will need to be done again.
- You can power-on virtual machines until you are out of slots, as a high reservation is set you will be severely limited!
Now you can imagine that “Host Failures” can be on the safe side… If you have 1 reservation set the math will be done with that reservation. This means that a single 10GB reservation will impact how many VMs you can power-on until HA screams that it is out of resources. But at least you are guaranteed you can power them on right? Well yes, but realistically speaking people disable Admission Control at this point as that single 10GB reservation allows you to power on just a couple of VMs. (16 to be precise.)
But but that beats Percentage Based right… because if I have a lot of VMs who says my VM with 10GB reservation can be restarted? First of all, if there are no “unreserved resources” available on any given host to start this virtual machine then vSphere HA will ask vSphere DRS to defragment the cluster.As HA Admission Control had already accepted this virtual machine to begin with, chances are fairly high that DRS can solve the fragmentation.
Also, as the percentage based admission control policy uses reservations AND memory overhead… how many virtual machines do you need to have powered-on before your VM with 10 GB memory reservation is denied to be powered-on? It would mean that none of the hosts has 10GB of unreserved memory available. That is not very likely as that means you would need to power-on hundreds of VMs… Probably way too many for your environment to ever perform properly. So chances of hitting this scenario are limited, extremely small.
Conclusion
Although theoretically possible, it is very unlikely you will end up in situation where one or multiple virtual machines can not be restarted when using the Percentage Based Admission Control policy. Even if you are using reservations on all virtual machines then this is unlikely as the virtual machines have been accepted at some point by HA Admission Control and HA will leverage DRS to defragment resources at that point. Also keep in mind that when using reservations on all virtual machines that Host Failures is not an option as it will skew your numbers as it does the math with “worst case scenario”, a single 10GB reservation can kill your ROI/TCO.
In short: Go Percentage Based!
Gopinath says
nice and great explanation for the end result of using % based and host failure based policy… i have also used % based with my client… now i have more clarity…no this..
Thanks Duncan… for taking time to explain…withe example..
Duncan Epping says
No problem
Peter Linkletter says
Thanks Duncan! I agree with you entirely when it comes to starting VMs but what about being sure that those VMs deliver their services at an end-user acceptable performance level? Do you have any information on that part of this equation?
Duncan Epping says
VC Ops can provide you details like that… Averages but I think also per VM. I am not the VC Ops expert, but check this article and leave a comment there. I am sure Hugo knows the answer:
http://www.virtualclouds.co.za/?p=526
Peter Linkletter says
I guess that is why I prefer a hot standby host. Not only do I know that my production VMs will run but that they will run well. Also, if you turn DVFS on, the power consumption will be minimal.
Now before you (and others) say “What a waste of a server!”, I would argue that it is no different than those hot-spare disks in the storage arrays. They too are wasted, until you need them (and then, they are precious)!
Billy Lucas says
Duncan, very good post. Just a couple of questions to clarify it. If I am not mistaken, it is better to just leave no reservations set because then you will not have the reservations of 10GB of RAM per VM? Also by no reservation you mean setting the percentage based limits on both CPU and RAM to 0%, correct?
I run a vCloud director 5.1.1 instance with a cluster that has 244GB of RAM, comprised of two hosts at the moment, hopefully adding a third soon. My original plan was to use host fail over in a “n-1” RAID5 like formula but thought that would be a waste of a host. So my next thought was to reserve 33% on each and shrink it as the cluster grew. Perhaps I’m just flat out wrong in my thinking and it would not honestly surprise me if I was. Thanks for any time or consideration you give.
Billy Lucas says
Ok, yeah I’m an idiot and shouldn’t be reading this type stuff late at night, scratch some of what I said about the 10GB of RAM reservation. I guess a more simplified version of what I asked would be. Would you recommend going percentage based with no reservation to ensure that you’ll power up all your VM’s and let vSphere HA do the math on what should be reserved on the respective hosts across the cluster?
Duncan Epping says
I would always go Percentage Based in ANY situation to be honest. In my opinion, it is the best option.
Billy Lucas says
So is it in your opinion best to leave the percentages at “0%” for both CPU and memory? If I understood the post correct.
Billy Lucas says
If you get a second, it would be MOST appreciated to let me know what percentage would be best to reserve. For CPU in memory typically. Is it better to have a lower reservation on them?
Duncan Epping says
I don’t know your workload or your SLA so it is difficult to say what your reservation should be. Typically in most environments people have no reservations set by default and only set them when certain application vendors request this. This is the approach I would generally advise to take. But once again, this fully depends on your Apps / SLA / Requirements so it may differ.
Jim Moyle says
Duncan,
I work for Atlantis Computing and I have a situation where I run into this every day. As I commonly have many VMs with > 60GB memory reservations this almost always results in the need to defragment.
The need to defragment has two issues, one in that it may take many multiple rounds of DRS to move the VMs out of the way delaying the restart, also that if the customerhas set affinity rules, DRS won’t break either reservation or affinity/anti-affinity rules to defragment. meaning there is a possibility the VM will not restart at all.
It might be an edge case, but for me the added risk and time inherent in deframentation make specifying a failover host the best way to go.
Jim Moyle
Duncan Epping says
In your scenario, and that is cornercase or at least solution specific, you have no other option indeed. Especially when you have a VM with a 60GB reservation then both “host failures” and “percentage based” will be unusable.
Great example Jim of when a “recommended practice” won’t work due to the solution. Thanks for your comment,
Jim Moyle says
Thanks Duncan, you might want to finish your sentence above to make your answer perfectly clear though 🙂