Yellow Bricks

by Duncan Epping

4.1

Maximum amount of FT virtual machines per host?

· ·

There was a discussion yesterday on our Socialcast system. The question was what the max amount of FT virtual machines was and what dictated this. Of course there are many things that will be a constraint when it comes to FT (memory reservations, bandwidth etc) but the one thing that stands out and not many realize is that the amount of FT virtual machines per host is limited to 4 by default.

This is currently controlled by a vSphere HA advanced setting called “das.maxftvmsperhost”. By default this setting is configured to 4. This advanced setting is an HA advanced setting (in combination with vSphere DRS) and defines the max amount of FT virtual machines, either primary or secondary or a combination of both, that can run  on a single host. So if for whatever reason you want a max of 6 you will need to add this advanced setting with a value of 6.

I do not recommend changing this however, FT is a fairly heavy process and in most environments 4 is the recommended value.

HA Admission Control the basics – Part 2/2

· ·

In part one I described what HA Admission Control is and in part two I will explain what your options are when admission control is enabled. Currently there are three admission control policies:

  1. Host failures cluster  tolerates
  2. Percentage of cluster resources reserved as failover spare capacity
  3. Specify a failover host

Each of these work in a slightly different way. And lets start with “Specify a failover host” as it is the most simple one to explain. This admission control policy allows you to set aside 1 host that will only be used in case a fail-over needs to occur. This means that even if your cluster is overloaded DRS will not use it. In my opinion there aren’t many usecases for it, and unless you have very specific requirements I would avoid using it.

The most difficult one to explain is “Host failures cluster tolerates” but I am going to try to keep it simple. This admission control policy takes the worst case scenario in to account, and only the worst case scenario, and it does this by using “slots”. A slot is comprised of two components:

  1. Memory
  2. CPU

For memory it will take the largest reservation on any powered-on virtual machine in your cluster plus the memory overhead for this virtual machine. So if you have one virtual machine that has 24GB memory provisioned and 10GB out of that is reserved than the slot size for memory is ~10GB (reservation + memory overhead).

For CPU it will take the largest reservation on any powered-on virtual machine in your cluster, or it will use a default of 32MHz (5.0, pre 5.0 it was 256MHz) for the CPU slot size. If you have a virtual machine with 8 vCPUs assigned and a 2GHz reservation then the slot size will be 2GHz for CPU.

HA admission control will look at the total amount of resources and see how many “memory slots” there are by dividing the total amount of memory by the “memory slot size”. It will do the same for CPU. It will calculate this for each host. From the total amount of available memory and CPU slots it will take the worst case scenario again, so if you have 80 memory slots and 120 CPU slots then you can power on 80 VMs… well almost, cause the number of slots of the largest hosts is also subtracted. Meaning that if you have 5 hosts and each of those have 10 slots for memory and CPU instead of having 50 slots available in total you will end up with 40.

Simple right? So remember: reservations –> slot size –> worst case. Yes, a single large reservation could severely impact this algorithm!

So now what? Well this is where the third admission control policy comes in to play… “Percentage of cluster resources reserved as failover spare capacity”. This is not a difficult one to explain, but again misunderstood by many. First of all HA will add up all available resources to see how much it has available. It will now subtract the amount of resource specified for both memory and CPU. Then HA will calculate how much resources are currently reserved for both memory and CPU for powered-on virtual machines. For CPU, those virtual machines that do not have a reservation larger than 32Mhz a default of 32Mhz will be used. For memory a default of 0MB+memory overhead will be used if there is no reservation set. If a reservation is set for memory it will use the reservation+memory overhead.

That is it. Percentage based looks at “powered-on virtual machines” and its reservation or uses the above mentioned defaults. Nothing more than that. No. it doesn’t look at resource usage / consumption / active etc. It looks at reserved resources. Remember that!

What do I recommend? I always recommend using the percentage based admission control policy as it is the most flexible policy. It will do admission control on a per virtual machine reservation basis without the risk of skewing the numbers.

If you have any questions around this please don’t hesitate.

HA Admission Control the basics – Part 1/2

· ·

Last week I received three different questions about vSphere HA Admission control and I figured I would lay out the basics once more. What is admission control?

vCenter Server uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection and to ensure that virtual machine resource reservations are respected.

Almost every thing you need to know about admission control is in that single sentence. But lets break it down in to more consumable bites:

  1. vCenter uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection.
  2. vCenter uses admission control to ensure that virtual machine resource reservations are respected.

So first and foremost… Admission control is not about resource management, I devoted a whole article to that so not going in to details, but HA admission control is all about reserving resources to allow for a failover.

Secondly, admission control ensures virtual machine resource reservations of powered-on VMs can be respected. This is because virtual machine resources reservations are required to be available in order for a power-on to successfully complete! Meaning that if you set a 5GB memory reservation there needs to be 5GB of unreserved memory available (+ reserved memory overhead) on a single host in order for this virtual machine to power-on. If that 5GB machine is actually actively using 40GB it might end up swapping / paging, as only those 5GB of reserved capacity is taken in to account!

Note the “+ reserved memory overhead”! Every virtual machine has a memory overhead. This is usually in the range of a couple hundred MBs. For a successful power-on attempt you will need to be able to reserve this memory. If there is not enough “unreserved memory capacity” the power-on attempt will fail. So in reality that 5GB could just be 5.15GB. Might seem irrelevant, but I will explain why it is relevant in a second. Did you spot the “powered-on”? Yes, admission control only takes the resource reservations of powered-on VMs in to account. So if you have a VM with a large memory reservation which is powered-off it will not impact your admission control calculations!

In summary:

  1. Admission Control is about reserving resources to allow for a fail-over.
  2. Admission Control is no resource management tool, it only takes reserved capacity of powered-on VMs in to account.

So now that you know what admission control is. There are three policies when it comes to admission control… and we will discuss these in Part 2 of this article.

How do I use das.isolationaddress[x]?

· ·

Recently I received a question on twitter how the vSphere HA advanced option “das.isolationaddress” should be used. This setting is used when there is the desire or a requirement to specify an additional isolation address. The isolation address is used by a host which “believes” it is isolated. In other words, if a host isn’t receiving heartbeats anymore it pings the isolation address to validate if it still has network access or not. If it does still have network access (response from isolation address) then no action is taken, if the isolation address does not respond then the “isolation response” is triggered.

Out of the box the “default gateway” is used as an isolation address. In most cases it is recommended to specify at least one extra isolation address. This would be done as follows:

  • Right click your vSphere Cluster and select “Edit settings”
  • Go to the vSphere HA section and click “Advanced options”
  • Add “das.isolationaddress0” under the option column
  • And add the “IP Address” of the device you want to use as an isolation address under the value column

Now if you want to specify a second isolation address you should add “das.isolationaddress1”. In total 10 isolation addresses will be used (0 – 9). Keep in mind that all of these will be pinged in parallel! Many seem to be under the impression that this happens sequential, but that is not the case!

Now if for whatever reason the default gateway should not be used you could disable this by adding the “das.usedefaultisolationaddress” to “false”. A usecase for this would be when the default gateway is a “non-pingable” device, in most scenarios it is not needed though to use “das.usedefaultisolationaddress”.

I hope this helps when implementing your cluster,

Want a free Kindle version of the VMware vSphere 4.1 HA and DRS technical deepdive?

· ·

Just a limited offer, two days only… The VMware vSphere 4.1 HA and DRS tech deepdive is free. WHAT? Yes it is free… $ 0,-. So hop over to amazon and pick up your free Kindle copy today!

For those who have been living under a rock, this book will explain the ins and outs of both vSphere HA and DRS. Admission Control, Resource Pools, Limits, HA Restart Timelines… it is all in there. Pick it up!