HA Admission Control the basics – Part 2/2

In part one I described what HA Admission Control is and in part two I will explain what your options are when admission control is enabled. Currently there are three admission control policies:

  1. Host failures cluster  tolerates
  2. Percentage of cluster resources reserved as failover spare capacity
  3. Specify a failover host

Each of these work in a slightly different way. And lets start with “Specify a failover host” as it is the most simple one to explain. This admission control policy allows you to set aside 1 host that will only be used in case a fail-over needs to occur. This means that even if your cluster is overloaded DRS will not use it. In my opinion there aren’t many usecases for it, and unless you have very specific requirements I would avoid using it.

The most difficult one to explain is “Host failures cluster tolerates” but I am going to try to keep it simple. This admission control policy takes the worst case scenario in to account, and only the worst case scenario, and it does this by using “slots”. A slot is comprised of two components:

  1. Memory
  2. CPU

For memory it will take the largest reservation on any powered-on virtual machine in your cluster plus the memory overhead for this virtual machine. So if you have one virtual machine that has 24GB memory provisioned and 10GB out of that is reserved than the slot size for memory is ~10GB (reservation + memory overhead).

For CPU it will take the largest reservation on any powered-on virtual machine in your cluster, or it will use a default of 32MHz (5.0, pre 5.0 it was 256MHz) for the CPU slot size. If you have a virtual machine with 8 vCPUs assigned and a 2GHz reservation then the slot size will be 2GHz for CPU.

HA admission control will look at the total amount of resources and see how many “memory slots” there are by dividing the total amount of memory by the “memory slot size”. It will do the same for CPU. It will calculate this for each host. From the total amount of available memory and CPU slots it will take the worst case scenario again, so if you have 80 memory slots and 120 CPU slots then you can power on 80 VMs… well almost, cause the number of slots of the largest hosts is also subtracted. Meaning that if you have 5 hosts and each of those have 10 slots for memory and CPU instead of having 50 slots available in total you will end up with 40.

Simple right? So remember: reservations –> slot size –> worst case. Yes, a single large reservation could severely impact this algorithm!

So now what? Well this is where the third admission control policy comes in to play… “Percentage of cluster resources reserved as failover spare capacity”. This is not a difficult one to explain, but again misunderstood by many. First of all HA will add up all available resources to see how much it has available. It will now subtract the amount of resource specified for both memory and CPU. Then HA will calculate how much resources are currently reserved for both memory and CPU for powered-on virtual machines. For CPU, those virtual machines that do not have a reservation larger than 32Mhz a default of 32Mhz will be used. For memory a default of 0MB+memory overhead will be used if there is no reservation set. If a reservation is set for memory it will use the reservation+memory overhead.

That is it. Percentage based looks at “powered-on virtual machines” and its reservation or uses the above mentioned defaults. Nothing more than that. No. it doesn’t look at resource usage / consumption / active etc. It looks at reserved resources. Remember that!

What do I recommend? I always recommend using the percentage based admission control policy as it is the most flexible policy. It will do admission control on a per virtual machine reservation basis without the risk of skewing the numbers.

If you have any questions around this please don’t hesitate.

Cloud Survey

I just received an email with the request to share the following. If you have a couple of minutes please fill out this survey. Your participation in this survey is extremely valuable to VMware and they sincerely appreciate your time in sharing information about your environment and needs.

We are conducting a small survey to understand how our customers are using different virtualization and cloud solutions to run the workloads in test/dev and production environment.  We would also like to learn more about what features and capabilities are more important in these environments and how much of time and resources each take.  This information will help us understand what matters most to our customers when it comes to different cloud solutions.

Please use below link to fill out the survey, it should not take more than 10 minutes of your time:



Performance of vCenter 5.0 in Remote Offices and Branch Offices (ROBO) white paper

I just finished reading the “Performance of VMware vCenter 5.0 in remote offices and branch offices (ROBO)” white paper. I thought it was an excellent read and recommend it to anyone who has a ROBO environment. Also it is interesting to know what kind of traffic hosts / VMs drive in general to vCenter. Especially the details around the statistics level are worth reading for those deploying larger environments as it also gives a sense of the amount of data that vCenter is processing.

Nice work Fei Chen! You can find the paper here:

Performance of VMware vCenter 5.0 in Remote Offices and Branch Offices (ROBO)
This document details the performance of typical vCenter 5.0 operations in a use case where vCenter manages ESXi hosts over a network with limited bandwidth and high latency, which is also known as a remote office, branch office (ROBO) environment.

(Although the date stamp on this entry says 2010 it is a June / 2012 paper, I will try to get this fixed!)

HA Admission Control the basics – Part 1/2

Last week I received three different questions about vSphere HA Admission control and I figured I would lay out the basics once more. What is admission control?

vCenter Server uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection and to ensure that virtual machine resource reservations are respected.

Almost every thing you need to know about admission control is in that single sentence. But lets break it down in to more consumable bites:

  1. vCenter uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection.
  2. vCenter uses admission control to ensure that virtual machine resource reservations are respected.

So first and foremost… Admission control is not about resource management, I devoted a whole article to that so not going in to details, but HA admission control is all about reserving resources to allow for a failover.

Secondly, admission control ensures virtual machine resource reservations of powered-on VMs can be respected. This is because virtual machine resources reservations are required to be available in order for a power-on to successfully complete! Meaning that if you set a 5GB memory reservation there needs to be 5GB of unreserved memory available (+ reserved memory overhead) on a single host in order for this virtual machine to power-on. If that 5GB machine is actually actively using 40GB it might end up swapping / paging, as only those 5GB of reserved capacity is taken in to account!

Note the “+ reserved memory overhead”! Every virtual machine has a memory overhead. This is usually in the range of a couple hundred MBs. For a successful power-on attempt you will need to be able to reserve this memory. If there is not enough “unreserved memory capacity” the power-on attempt will fail. So in reality that 5GB could just be 5.15GB. Might seem irrelevant, but I will explain why it is relevant in a second. Did you spot the “powered-on”? Yes, admission control only takes the resource reservations of powered-on VMs in to account. So if you have a VM with a large memory reservation which is powered-off it will not impact your admission control calculations!

In summary:

  1. Admission Control is about reserving resources to allow for a fail-over.
  2. Admission Control is no resource management tool, it only takes reserved capacity of powered-on VMs in to account.

So now that you know what admission control is. There are three policies when it comes to admission control… and we will discuss these in Part 2 of this article.

Creating a nested lab

I was just building a nested lab to record some demo videos. I find myself googling for this every single time so I figured I would write about it so I can easily get it of my own website. Many have written about this before and all credits go to William Lam and Eric Gray, which are the two  main blogs I have used in the past to get this working.

After installing ESXi on my physical box I “ssh” in to it. In order to allow “nested ESXi” to boot a 64bit OS you will need to run the following:

echo 'vhv.allow = "TRUE"' >> /etc/vmware/config

After you have done that you will want to make sure you will get network connection. Go to your “VM Network” portgroup, or if you named it differently the portgroup that is used to connect the virtual ESXi hosts to. For each of the hosts do the following:

  1. Click on the host
  2. Go to “Configuration”
  3. Click on “Networking”
  4. Click “Properties” on the vSwitch
  5. Select the correct portgroup
  6. Click “Edit”
  7. Click “Security”
  8. Set “Promiscuous Mode” to “Accept”
  9. Click “Ok”
  10. Click “Close”

Now for each virtual ESXi host (note there is a “guest os” called ESXi 5 in there, use it!) that you have created do the following:

  1. Right click on the VM
  2. Click “Edit settings”
  3. Click the “Options” tab
  4. Click on “CPU/MMU virtualization”
  5. Select the 4th option “Use Intel VT-x / AMD-v…”

I am building this out to record a new of “DR of the Cloud”. In other words, 3 virtual clusters + vCloud Director + SRM + vSphere Replication + Virtual Storage Appliances… Cool stuff right.

Guest post: Chasing Virtualization News Has Never Been Easier

As Dell was kind enough to list me in the Virtualization Connect Widget I figured it would be fair to share this with all of you. One thing I want to point out is that I am currently not using the plugin myself as it didn’t fit my layout. I do believe this is a really cool initiative and if I re-do my layout I will try to make room for it.

Chasing Virtualization News Has Never Been Easier

By: Matthew Smith

Every day there are 7,330 blog posts written about the topic of virtualization. If your schedule is anything like mine you won’t be able to even read 1% of those blog posts. Think of all the important information you might miss out on? In answer to this problem the Dell virtualization team has been working hard to help make your life easier.

Introducing the Virtualization Connect Widget. The first widget to bring the latest virtualization news directly to your website. The virtualization connect widget is a WordPress plugin that can be installed on any WordPress blog (other platforms coming soon). The widget is a content aggregator that pulls summary descriptions and titles from the top virtualization bloggers in the industry (including Yellow-Bricks.com).

Here’s How the Widget Works

  • The widget has a special crawler that goes to a pre-set list of virtualization bloggers that were selected based on quality, knowledge, and freshness of content.
  • The crawler then takes a brief summary and title of the latest article and brings it back to the website that installed the widget.
  • Articles within the widget are then stack ranked based on the total amount of social shares it received. For example if Duncan’s article received 50 tweets and someone else’s article within the widget received 25 tweets, Duncan’s article would outrank the other. This helps the installer of the widget to see the top stories within virtualization news.

Customization of the Widget

  • Each widget is fully customizable to each installer’s individual needs. Within the widget settings you can actually select which virtualization thought leaders you want to follow.
  • You can also add your own blogs and RSS feeds you would like to follow. Think of the widget as a super charged blogroll.

We are always looking to add additional thought leaders to the widget. If you think your blog has the right stuff then leave a comment or visit this page and submit your blog on the left side of the page.

Win a free VMworld ticket by sharing your vC Ops story update…

A few weeks back I wrote a blog entry on a “Cool Contest: Tell your vC Ops Story”, and since then we have had some really great submissions but of course we can always use more. As a member of the judging panel I thought I’d share a few thoughts on those who may be interested in entering, but may be hesitant for other reasons.

  1. “What if my story is not good enough”? Don’t let this deter you from entering, you’d be surprised at how success stories can actually seem routine in your own day to day, but in the grander scheme of things prove invaluable to others that may not have the same insight as you.
  2. “I want the ticket to VMworld, but I am not a seasoned presenter”, This is your opportunity to shine!  That is why the panelists (me included) and the VMware team will both help you both prepare (slideware, visuals and dry runs) and present side by side with you – making the experience all the more enjoyable and fruitful!

There is also a good blog on the VMware blog site that provides more guidance on how to enter and advice to what the judges will be looking for.  One last piece of advice, do not put off your entry any longer – the contest runs until July 11th, and the anticipated “mad rush” at the end will only lump your entry with those trying to make the deadline, so just enter now!