Yellow Bricks

by Duncan Epping

u2

HA isolation response “shutdown guest”

· ·

So if you’re like me, better safe than sorry… than you’ve probably set your ESX 3.5 U2 HA cluster to “shutdown VM” instead of “Power off VM” or “Leave VM powered on”. By now most of you probably already noticed that when an isolation occurs HA will allow the VM to shutdown clean within 5 minutes. When the 5 minutes are past HA will shutdown the VM’s no matter what.

But for some of you 5 minutes(300 seconds) might just not be long enough, or if you have an ultra fast environment 5 minutes might just be to damn long….

So what can you do to shorten or extent… It’s easy, open up you HA cluster settings and click on advanced options and add the following:

das.isolationShutdownTimeout – values in seconds, default is 300

I’ve also updated my HA advanced settings blog! If anyone has more advanced settings that aren’t on the list let me know!

Response on the DABCC VMware HA vs Citrix HA article

· ·

I clearly don’t know much about Citrix version of HA, but I do know a thing or two about VMware’s version of HA.

The following are outtakes of the article over at DABCC:

VMware’s HA is heavily dependent on DNS or alternatively hosts entries being in place. The VMware implementation is based on the Legato Automated Availability Management (AAM), in fact some of us will recall that it used to place those logs into /opt/LGTOaam512/logs/ (since 3.5 this has been moved /opt/vmware/aam).

VMware’s HA uses the network to establish a heartbeat between all the ESX Hosts participating, So practically, what does this mean to the poor bloke who has to support the servers? If you network has a bit of a flap (personally I always blame the Network guysJ), your servers will implement an “isolation response”, the default server response will shut down your Virtual Machine to release the shared storage locks, this will allow the machine to be restarted on another host, this of course may not be desirable if the server is busy doing something, i.e. you may cause corruption or other issues with the Application/Database. In other words it won’t perform a clean shutdown. This is configurable such that you can keep the machines powered on, but this isn’t recommended in the case of NAS or iSCSI (as they are also network dependant) and you may end up with a split-brain situation.

There is now also experimental support for component level HA, i.e. if a Virtual Machine fails, then VMware will try to restart it.

  1. As of ESX 3.5 U2 High Availability doesn’t heavily lean on DNS anymore, it gets its hostname and ip info from VirtualCenter.
  2. ESX 3.5 U2 gives you the possibility to cleanly shutdown a VM in case of an isolation.
  3. Normally one would indeed provide it’s SC with redundancy, and preferably via two separate switches to avoid the problems you are describing.
  4. Virtual Machine High Availability isn’t experimental anymore as of ESX 3.5 U2.

Starting VM’s problem with 3.5 U2

· ·

As everyone probably already knows by now there’s a problem with 3.5 U2.  VMware is working on a patch as we speak. There has been a KB article released, but it seems like everyone is clicking on the same link at the same moment cause it’s hard to get a decent respond.

The error message that appears:

This product has expired. Be sure that your host machine’s date and time are set correctly.
There is a more recent version available at the VMware web site: http://www.vmware.com/info?id=4.
————–
Module License Power on failed

In short, the workaround is simple just set the date back and you will be able to power on the VM’s again, it would be smart to set the time to correct value again as soon as you started the VM. As soon as I know more about the new 3.5 U2 update I’ll let you guys know!

And a nice work around from the VMTN forum:

Find the host where a VM is located
run ‘ vmware-cmd -l ‘ to list the vms.
issue the commands:
service ntpd stop
date -s 08/01/2008
vmware-cmd /vmfs/volumes/vm path/vmname.vmx start
service ntpd start

das.allowNetwork where and when

· ·

I’ve seen a lot misinterpretations of the new advanced HA option “das.allowNetwork”. What is this option for and when should I use it or shouldn’t is the most asked question.

So let’s start with a little history, back in the days “pre 3.5 U2” one could configure a cluster on multiple IP subnets without facing any problems, but this could lead into start up or execution problems. HA would configure without checking if the network on every host was compatible with the other, but in ESX 3.5 U2 this procedure changed. As of this version during the configuration of HA the host checks it’s network for compatability with the first configured HA host. So if the first host has the following configuration:
esx01 – 192.168.1.11 – 255.255.255.0.

And the second host has the following configuration:
esx02- 192.168.2.11 – 255.255.255.0

HA will fail with the following error:
“HA Agent on <hostname> in cluster <clustername> in <datacenter> has an error Incompatible HA Networks: Host has network(s) that don’t exist on cluster members: <ip address>: Cluster has network(s) missing on host: <ip address>: Consider using Advanced Cluster Settings das.allowNetwork to control network usage”

With the advance option “das.allowNetwork” you can rule out certain Service Consoles for HA usage. In other words, if you have several SC’s on each ESX host you can specify which SC should be used for HA. HA, as most of you probably know, uses the SC network for it’s heartbeat.

A couple of examples when to use das.allowNetwork and when not to use it:

  1. So what should you do to get things running when you’ve used different subnets for each host?
    Well that’s an easy one… get all your Service Consoles on the same subnet. Even if’s a routable vlan, HA will just calculate the network and if the values mismatch than configuring HA will fail!
  2. So what if I’ve got multiple SC’s and some are on different subnets but there’s at least one on each host on the same subnet with the same portgroup name?
    Here’s where das.allowNetwork comes in to place, set this option in the “Advanced Option” section of the HA tab and specify which SC should be used for HA. The ones that aren’t specified will not be used.

There have been numerous people asking if this “network compatibility” check could be disabled. At this moment the answer is no and there’s now workaround at this moment. When using das.allowNetwork specify the first as “das.allowNetwork0” the second as “das.allowNetwork1” etc.

More info can be found here:

KB1006541

KB1006606

Resource Management PDF

What if my VirtualCenter server crashes?

· ·

Seva, a VMware Technical Account Manager, put together a cool table with the implications of a VirtualCenter crash. This is a follow up to my blog about VirtualCenter getting more important by the minute. I think the most important thing to remember is that the VM’s keep running whatever happens to your VC Server and HA will still work if VC fails, well except for adding hosts to the cluster of course. So reinstalling the VirtualCenter server and re-adding the hosts is still possible, but in my opinion not recommended. Especially when you’ve got complex Resource Pools and Folder structures set up.

Open this link to the PDF or click on the picture below.