ha

I had a question about HA Admission Control today and as this is something that has come up multiple times I figured I would dedicate an article to it. This customer had enabled HA Admission Control and artificially wanted to control the amount of virtual machines a single host could run by manually specifying the slot size. (For more details on Admission Control slot sizes and how to configure these read the Deepdive page.) When they simulated a failure they were surprised that some host had more virtual machines running than should be allowed according to the configured slot size… This is however, contrary to their beliefs, by design. Let me copy/paste a paragraph from our book which talks about admission control.

What is HA Admission Control about? Why does HA contain this concept called Admission Control? The “Availability Guide” a.k.a HA bible states the following:

“vCenter Server uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection and to ensure that virtual machine resource reservations are respected.”

Please read that quote again and especially the first two words. Indeed it is vCenter Server that is responsible for Admission Control. Although this might seem like a trivial fact it is important to understand that this means that Admission Control will not disallow HA initiated restarts. HA initiated restarts are done on a host level and not through vCenter. It is Admission Control’s task to ensure sufficient resources are available for HA to restart virtual machines, hence the reason HA does not take Admission Control in to account.

I hope this clears things up. I was pretty sure I have discussed this in multiple articles but as it comes up fairly often I figured dedicating and article to it would make it easier to find. I know it is not really clear in our documentation and I’ve requested this to be changed to reflect the actual behavior and avoid misunderstandings like these.

Bilal Hashmi wrote a nice article about HA today and in this article he asked a couple of questions. As I think the info is useful for everyone I decided to respond through a blog article instead of by commenting.

Let me start by saying that in general HA should never be disabled. The later versions of vSphere have a neat option called “Enable Host Monitoring”. This option should be used for scheduled network maintenance. The difference between disabling host monitoring and disabling HA is that disabling host monitoring does not cause a full reconfiguration (see screenshot below) of HA and a new election process. Just the “host monitoring” functionality is disabled, which is what you want in this scenario.

Bilal asked multiple questions / made multiple statements in his article, I will respond to two of these specifically to explain the way HA handles failures/isolation:

In this case within 30 sec of the management network outage, each host would have declared itself isolated and wont attempt to restart any VMs like the primaries would in vSphere 5.

So why is this? As soon as a Master is isolated it will drop “ownership” of datastores on which VMs are running that are part of its cluster. Before the other hosts trigger the isolation response for a given VM they will validate if the datastore on which this VM is stored is “owned” by a master. In the case of a cluster wide isolation due to a network outage / maintenance the ownership would be dropped and this would result in HA not triggering the isolation response. This is a major change compared to vSphere 4.x and prior!

Now what happens when the network outage is over and the hosts are in a position to talk to each other? I have not been able to find documentation on whether an isolated host will enter an election (vSphere 4 or 5) ones the communication channel is open and bring the cluster back to life.

Lets focus on vSphere 5.0 as that seems most relevant. A host remains isolated until it observes HA network traffic, like for instance election messages OR it starts getting a response from an isolation address. Meaning that as long as the host is in “isolated state” it will continue to validate its isolation by pinging the isolation address. As soon as the isolation address responds it will initiate an election process or join an existing election process and the cluster will return to a normal state.

There’s absolutely no need to manually intervene. HA takes care of all of this for you.