I received this question last week about a recommendation which was in the vSphere 5.1 Hardening Guide. The recommendation in the vSphere 5.1 Hardening Guide is the following:
By default, all virtual machines on an ESXi host share the resources equally. By using the resource management capabilities of ESXi, such as shares and limits, you can control the server resources that a virtual machine consumes. You can use this mechanism to prevent a denial of service that causes one virtual machine to consume so much of the host’s resources that other virtual machines on the same host cannot perform their intended functions.
Now it might be just me but I don’t get the recommendation and my answer to this customer was as follows:
Virtual machines can never use more CPU/Memory resources then provisioned. For instance, when 4GB of memory is provisioned for a virtual machine the Guest OS of that VM will never consume more than 4GB. Same applies to CPU, if a VM has a single vCPU than that VM can never consume more than a single core of a CPU.
So how do I limit my VM? First of all: right sizing! If your VM needs 4GB then don’t provision it with 12GB as it some point it will consume it. Secondly: shares. Shares are the easiest way to ensure that the “noisy neighbor” isn’t pushing away the other virtual machines. By even leaving the shares set to default you can ensure that at least all “alike VMs” have more or less the same priority when it comes to resources. So what about limits?
Try to avoid (VM Level) limits at all times! Why? Well look at memory for a second, lets say you provision your VM with 4GB and limit it to 4GB and now someone changes the memory to 8GB but forgets to change the limit. So what happens? Well your VM uses up the 4GB and moves in to “extra 4GB” but the limit is there, so you the VM will experience memory pressure and you will see ballooning / swapping etc. Not a scenario you want to find yourself in right, indeed! What about CPU then? Well again, it is a hard limit in ALL scenarios. So if you set a 1GHz scenario but have a 2.3GHz CPU, your VM will not consume the 2.3GHz ever…. A waste? Yes it is. And not just VM level limits, there is also an operational impact with resource pool limits.
I can understand what the hardening guide is suggesting, but believe me you don’t want to go there. So let it be clear, AVOID using limits at all times!