Yellow Bricks

by Duncan Epping

ESX

DRS Sub Cluster? vSphere 4.next

· ·

On the community forums a question was asked around Campus Clusters and pinning VMs to a specific set of hosts. In vSphere 4.0 that’s currently not possible unfortunately and it definitely is a feature that many customers would want to use.

Banjot Chanana revealed during VMworld that it was an upcoming feature but did not go into much details. However on the community forums, thanks @lamw for point this out, Elisha just revealed the following:

Controls will be available in the upcoming vSphere 4.1 release to enable this behavior. You’ll be able to set “soft” (ie. preferential) or “hard” (ie. strict) rules associating a set of vms with a set of hosts. HA will respect the hard rules and only failover vms to the appropriate hosts.

Basically DRS Host Affinity rules which VMware HA adheres to. Can’t wait for the upcoming vSphere version to be released and to figure out how all these nice “little” enhancements change our designs.

HA: Max amount of host failures?

· ·

A colleague had a question around the maximum amount of host failures HA could take. The availability guide states the following:

The maximum Configured Failover Capacity that you can set is four. Each cluster has up to five primary hosts and if all fail simultaneously, failover of all hosts might not be successful.

However, when you select the “Percentage” admission control policy you can set it to 50% even when you have 32 hosts in a cluster. That means that the amount of failover capacity being reserved equals 16 hosts.

Although this is fully supported but there is a caveat of course. The amount of primary nodes is still limited to five. Even if you have the ability to reserve over 5 hosts as spare capacity that does not guarantee a restart. If, for what ever reason, half of your 32 hosts cluster fails and those 5 primaries happen to be part of the failed hosts your VMs will not restart. (One of the primary nodes coordinates the fail-over!) Although the “percentage” option enables you to save additional spare capacity there’s always the chance all primaries fail.

All in all, I still believe the Percentage admission control policy provides you more flexibility than any other admission control policy.

Storage IO Control, the movie

· ·

Not sure why hardly anyone picked up on this cool youtube movie about Storage IO Control(SIOC), but I figured it was worth posting. SIOC is probably one of the coolest version coming to a vSphere version in the near future. Scott Drummonds wrote a cool article about it which shows the strength of SIOC when it comes to fairness. One might say that there already is a tool to do it and that’s called per VM disk shares, well that’s not entirely true… The following diagrams depict the current situation(without…) and the future(with…) :

As the diagrams clearly shows, the current version of shares are on a per Host basis. When a single VM on a host floods your storage all other VMs on the datastore will be effected. Those who are running on the same host could easily, by using shares, carve up the bandwidth. However if that VM which causes the load would move a different host the shares would be useless. With SIOC the fairness mechanism that was introduced goes one level up. That means that on a cluster level Disk shares will be taken into account.

There are a couple of things to keep in mind though:

  • SIOC is enabled per datastore
  • SIOC only applies disk shares when a certain threshold(Device Latency, most likely 30ms) has been reached.
    • The latency value will be configurable but it is not recommended for now
  • SIOC carves out the array queue, this enables a faster response for VMs doing for instance sequential IOs
  • SIOC will enforce limits in terms of IOPS when specified on the VM level
  • No reservation setting for now…

Anyway, enough random ramblings… here’s the movie. Watch it!

For those with a VMworld account I can recommend watching TA3461.

Which host is selected for an HA initiated restart?

· ·

Got asked the following question today and thought it was valuable for everyone to know the answer to this:

How is a host selected for VM placement when HA restarts VMs from a failed host?

It’s actually a really simple mechanism. HA keeps track of the unreserved capacity of each host of the cluster. When a fail-over needs to occur the hosts are ordered. The host with the highest amount of unreserved capacity being the first option. Now to make it absolutely crystal clear, HA keeps track of the unreserved capacity and it is not DRS which does this. HA works completely independent of vCenter and as we all know DRS is part of vCenter. HA also works when DRS is disabled or unlicensed!

Now one thing to note is that HA will also verify if the host is compatible with the VM or not. What this means is that HA will verify if the VMs network is available on the target host and if the datastore is available on the target hosts. If both are the case a restart will be initiated on that host. To summarize:

  1. Order available host based on unreserved capacity
  2. Check compatibility (VM Network / Datastore)
  3. Boot up!

vSphere Update 2 released

· ·

By now the whole world has probably read that vSphere 4 Update 2 has been released. (release notes vCenter, release notes ESX, release notes ESXi ) Some of you might have even started slowly upgrading their test systems. (Like I am doing at the moment…)

I will not copy the full release notes but I do want to point out a couple of things on which I have been waiting for.

What’s Cool:

  • vSphere 4.0 U2 includes an enhancement of the performance monitoring utility, resxtop. The resxtop utility now provides visibility into the performance of NFS datastores in that it displays the following statistics for NFS datastores: Reads/s, writes/s, MBreads/s, MBwrtn/s, cmds/s, GAVG/s (guest latency).
  • VMware High Availability configuration might fail when advanced HA option das.allowNetwork uses vNetwork Distributed Switch (vDS) port group on an HA-enabled cluster, if you specify a vDS port group by using the advanced HA configuration option das.allowNetwork, the HA configuration on the hosts might fail. This issue is resolved in this release. Starting with this release, das.allowNetwork works with vDS.
  • The esxtop and resxtop utilities do not display various logical cpu power state statistics; this issue is resolved in this release. A new Power screen is accessible with the esxtop utility (supported on ESX) and resxtop utility (supported on ESX and ESXi) that displays logical cpu statistics. To switch to the Power screen, press y at the esxtop or resxtop screens.
  • For devices using the roundrobin PSP the value configured for the –iops option changes after ESX host reboot. If a device that is controlled by the roundrobin PSP is configured to use the –iops option, the value set for the –iops option is not retained if the ESX Server is rebooted. This issue is resolved in this release.

In this release many issues have been fixed, but also some new features have been added. For me personally the first one in the list is important. Up to ESX 4 Update 1 it was always needed to dive in to vscsiStats to see the Guest Latency for NFS based storage. As of Update 2 you can just run esxtop and check the statistics for your NFS datastore. This will definitely simplify troubleshooting, single pane of glass!