I just wanted to write a couple of lines about Resource Pools. During most engagements I see environments where Resource Pools have been implemented together with shares. These Resource Pools are usually labeled “Low”, “Normal” and “High” with the shares set respectively. This is the traditional example being used during the VMware vSphere / VI3 course. Why am I writing about this you might ask yourself as many have successfully deployed environments with resource pools.
The problem I have with default implementations is the following:
Sibling resource pools share resources according to their relative share values.
Please read this line a couple of times. And then look at the following diagram:
What’s the issue here?
RP – 01 -> 2000 Shares -> 6 VMs
RP – 02 -> 1000 Shares -> 3 VMs
So what happens if these 9 VMs start fight for resources. Most people assume that the 6 VMs, which are part of RP-01, get more resources than the 3 VMs. Especially when you name them “Low” and “Normal” you expect the VMs which are part of “Low” to get a “lower” amount of resources than those which belong to the “Normal” resource pool. But is this the case?
No it is not. Sibling resource pools share resources according to their relative share values. In other words, resources are divided on a resource pool level, not on a per VM level. So what happens here? RP-01 will get 66% of the resources and RP-02 will get 33% of the resources. But because RP-01 contains twice as many VMs as RP-02 this will not make a difference when all VMs are fighting over resources… Each VM will roughly get the same amount of processor time. This is something that not many people take into account when designing an infrastructure or when implementing resource pools.
“RP-01 will get 66% of the resources and RP-01 will get 33% of the resources”
i think you want say :
“RP-01 will get 66% of the resources and RP-02 will get 33% of the resources”
so I should set the shares on the VM self to High instead of a (high, normal/low) resource pool? or how do you differnce your VM’s?
In a small environment (not a lot of hosts or VMs), doesn’t it make sense to just have one resource pool for all VMs, then just assign high, normal, low shares per VM??…and this could take care of things appropriately??
Duncan –
SIMPLE! SIMPLE! SIMPLE!
Maybe they can add this to the course?
Dave
@Tom: Why create a resource pool at all in a small environment? What will it add?
I’ll try to put this to good use immideately.
Sounds like a PowerShell script could really fit the bill here to pump out a quick summary/audit of the # of shares dsitributed to each VM Guest per Resource Pool created…
Shares are only used when there not enough resources available.
I always leave them default. I just use reservations.
I know Rob, but when they are used it can lead to unexpected results….
Could anybody please answer my question :))???
so I should set the shares on the VM self to High instead of a (high, normal/low) resource pool? or how do you differnce your VM’s?
If you set VM shares within a resource pool than those will only be used between VMs within the same resource pool. So if you look at the diagram above the first six will fight amongst each other and the second three will. Not against each other.
@duncan from tom: I didn’t mean *add* a resource pool, I meant not making a new RP at all and just setting the hosts’ shares to low, normal, high. My dev/test servers all get a ‘low’ share and the rest all get ‘normal’ shares. I don’t make any resource pools, I assume the whole host is its own resource pool. Is this proper??
Thank you for this post, you opened my eyes, I was using the exact WRONG config you talked about :((
Resource pools are very useful for VISUAL separation of virtual machines in categories.
If you manage shares properly taking into account the number of virtual machines this should work good.
Duncan,
Thanks for this post. I have worked with numerous customers and always recommended this resource pool design as a starting point. I didn’t really take into account this scenario though.
The main thing to remember with resource pools, shares, reservations, limits, etc. at ANY level, is the fact that there are soooo many different ways to configure clusters and there really isn’t any one perfect way to do it. It really depends on the environment, what your VMs are used for, what you are trying to accomplish, if you have SLA’s…. So many things to consider. That being said, thanks for bringing this to my attention. something I will definitely consider in future configurations.
I was just thinking of something else. Would this configuration with shares make sense in an environment where you had a more even spread of vms across resoirce pools? For example, say you had 6 vms in each pool. Then shares would make sense here. Thoughts?
Yes if they are evenly divided it would make more sense indeed.
@Tom : yes if you do it on a per VM basis it would solve this issue. But that’s a lot of work isn’t it?!
So then the proper way to do a simple “High” “Normal” “Low” would be to start with High and inside High put Normal and inside Normal put Low… right?
That way the High VMs always get resources… after those normal do… and after normal the lows do?
Sorry I don’t understand the question. Sibbling fight amongst each other for resources. So even if you create a parrent with multiple children it will not make a difference. The key is correct share values.
I was looking at my customer’s environment today and they are sort of mixed. They have multiple clusters but all configured with High and Low resource pools. One pool for production(high shares), the other for Staging(integration; low shares). A couple of their clusters they have way more Production VMs than they do Staging VMs, so they could run into the above scenario if the VMs start to contend for resources. The other clusters are fairly balanced so the pools should work fairly efficiently. On the unbalanced clusters, my guess is I could increase the share amount for the Production cluster so that it is relatively larger than the Staging pool. For example, if the pools were originally a 4:1 ratio(High:Low), I could increase the custom share and make it say a 8:1 ratio. That way, the Production pool would get a higher overall percentage of the available resources. If my thinking is correct.
My conclusion after reading your article: Recount the vm’s in the Resource Pools in a HA-cluster on separated times and reconfigure your shares if necessary. Is this a correct conclusion?
Correct,
another option is to leave shares alone 🙂
Duncan, CPU and Memory shares are simple to understand.
But what about Disk shares – what exactly is measured? IOPS, bandwidth? I.e. if I have VM generating enormous amount of short operations how can I set disk IO limit for IT so it won’t affect other VMs?
Interesting indeed. Is there an easy way to accomplish what people (me including..) think they’re getting with three resource pools set to high, low and normal shares? It would be nice to atleast seperate test machines from production without having to set individual VM resources.
With regards to Chris R.’s comment. This seems logical to me, except that I wouldn’t set shares at all. If you have:
High Priority VMs RP
|- Medium Priority VMs RP (A nested Pool within High Priority)
|- Low Priority VMs RP (A nested Pool within Medium Priority)
Without any shares set, would this not achieve the desired result, sort of. If you had 8Vms in High and 8VMs in Medium nested pool, then in a contensious situation, would the High Priority VMs get way more priority than the VMs within the Medium priority, and thusly the medium prioirty VMs get way more resources than the VMs in the low priority pool?
That still depends on the amount of VMs in the Low Priority Pool vs the amount of VMs in the Medium Priority pool. The High Priority Pool will get more resources indeed. tough question, had me thinking for a while.
Excellent post. We have this same problem in our environment and are getting ready to try and redesign our resource pools because of it.
Duncan, would just creating more ‘normal priority’ resource pools be a viable fix for this? We have no chargeback in our environment and we really have no reason to set resource pools to anything other than ‘high’, ‘medium’ or ‘low’. We like them divided in this way but want to ensure the problem you note in this post doesn’t happen. One way of doing this seems to be simply creating more “normal priority” pools and dividing the “normal priority” vm’s among those pools. Thoughts?
The conclusion I come to from this discussion is.
1. Most of use incorrectly configured resource pools possibly to the detriment of our guests performance.
2. There is no point creating resource pools in most environments unless you are going to monitor the number of guests in them and adjust the shares accordingly (and/or unless your using them for security reasons)
3. Given the fact that most people don’t get this right Resource pools are fundamentally flawed by DESIGN. Good one VMWare.
4. What we want from VMware is a resource pool solution designed so it allows us to effectively apply shares to a guest not a pool. I guess you could say something of a policy type solution. Eg.Put the guest into the resource pool and it inherits the pool settings rather than shares the pool settings.
Again may I say “GOOD ONE VMWARE. GREAT DESIGN!”
Again may I say “GOOD ONE VMWARE. GREAT DESIGN!”
Again may I say “GOOD ONE VMWARE. GREAT DESIGN!”
Lets all put this one to VMWare as a feature request.
Hey Duncan,
Thanks for the nice article.
So, I was thinking it’s all about the following number:
Shares/# of VM’s
Show for RP-01 this number comes out to be 2000/6 = 333
& for RP-02 this number would be 1000/3 = 333
So we can safely say that when all VM’s are contending all of them would get exactly the same amount of resources.
The higher that number is, higher is the resources per VM.
Am I over simplifying things? 🙂
Is the share applied at VM or vCPU level? If I have 2 VM in a RP and I use default Normal share for both. But VM1 has 4 vCPU and VM2 1 vCPU. During resource contention, will VM1 get 4 times the CPU resource than VM2?
It’s applied at the VM or RP level. What’s interesting is this that when your host has resource contention, your 4 vCPU VM will suffer more as it will have to wait till 4 cpu can be scheduled for it to do anything.. while it will probably be easier for the 1vCPU VM to sneak in and out.
You could have a VM with 8 vCPU that may still have less shares than the a VM with 1 vCPU. Number of vCPU will not dictate the amount of shares. But keep in mind more vCPU also means more obstacles for scheduling during contention. I am usually conservative in giving away vCPUs.
Ok please disregard my earlier response, I had to go back and rethink about this as Duncan pointed out it wasnt accurate. I think the following is what will hold true.
If you have a 2 vCPU VM and 1 single vCPU VM. In the resource allocation section you will notice that the dual vCPU VM will have twice as many shares. So during contention, the single vCPU VM will suffer as it has half as many shares.
So in your example, you are right. VM with 4 vCPU will have 4 times more shares than the one with a single CPU. So the 4vCPU VM will have a bigger slice when it comes to CPU contention time.
credit:
Duncan and Frank HA/Deepdive AND
http://steveshomelab.wordpress.com/2011/05/27/too-many-vcpus-a-simple-demonstration/
Question: What happens to the total number of shares provided to a resource pool by the Parent pool (%shares) if a virtual machine is removed from the pool?
My guess is that they stay the same.. but not understanding the mechanics/math to be 100% sure…?
At first Thanks.
Question:How can we share my virtual machine pool share resource one virtual machine to another virtual machine ,when our one virtual machine off and than this machine resource share other machine access this pool resource is it possible.? please answer me.
Thanks
Aga Kamruzzaman Chowdhury.
Phone +8801913-022678
mail : [email protected]
Duncan, way to put it simply.
My thoughts on Resource Pool Shares is to always keep a balance, this (in my opinion) can not be done using the default shares. I like to create my own custom shares and update them as VM’s are introduced in to the environment (I know it’s more work, but I am a bit OCD)
Here is the basic formula that I have always used, I would love to hear your opinion on them.
First, I create my own custom values. High=4 Normal=2 Low=1 (You can make this what ever you want) , I then create my Parent resource pools RP-High, RP-Normal, RP-Low… So working from the bottom up, Let’s say I have 40 VM’s in High, 50 in Nomal and 60 in Low, using the basic equation – [#ofVM’s * RPValue]=Shares So, for High the math would be [40*4]=160 Shares … Normal [50*2]=100 Shares… Low [60*1]=60 Shares This would (in my mind) evenly distrubute shares amoungs VM’s in their respected RP’s.
I also like to break down my VM’s in to child RP’s to account for VM’s with multiple vCPU’s, this is the simple equation that I would use for that (I will only do the High RP to keep this short)
‘[#ofVM’s * RPValue] * #ofVCPU’s = Shares’ so the Math would be (for an RP of VM’s with 4 vCPU’s=20 and 2 vCPUs = 20) RP-vCPU4 – [20*4]*4=320 Shares — RP-vCPU2 – [20*4]*2=160 Then I would add the two values of my child RP’s (working from the bottom up) together, this would give me the total number of shares to assign to my Parent RP (RP-High = 480 Shares)
I would love to hear your thoughts on doing it this way.
Hi Duncan,
Need some clarification, here is the snippet from the vsphere 5.1 documentation center.
(refer below link for scenario picture)
http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.wssdk.pg.doc%2FPG_Resources.15.5.html
((Now, consider another scenario with VM-M1 and VM-M2, shown in Admission Control with Expandable Resource Pools, Scenario 2.
■
Power on two virtual machines in RP-MOM with a total reservation of 3GHz.
■
You can still power on VM-K1 in RP-KID because 2GHz are available locally.
■
When you try to power on VM-K2, RP-KID has no unreserved CPU capacity so it checks its parent. RP-MOM has only 1GHz of unreserved capacity available (5GHz of RP-MOM are already in use—3GHz reserved by the local virtual machines and 2GHz reserved by RP-KID). As a result, you cannot power on VM-K2, which requires a 2GHz reservation. ))
**My questions is as per the second example scenario, all the 4 VMs should power ON right ? If there was 3rd VM in kid resource pool then the 3 VM would not power on.
Kindly help me clarifying this.