Yellow Bricks

by Duncan Epping

u2

VMware HA implementations notes

· ·

As most of you know I work for VMware which means I receive a huge amount of email and documents with excellent technical info. Some I can’t blog about but some I can. Last week I received this VMware HA implementations notes document by Seva Semouchin. Seva is a Technical Account Manager based in Germany!

This document deals about VMware HA clusters and all the advanced options you have with the use cases for these options. And that’s what makes this document very useful, the use cases! So everyone dealing with HA clusters should definitely read this document. Thanks Seva for letting me distribute it, and thanks for a great document!

Weird problems with enabling HA on ESXi

· ·

A couple of days ago an ex-colleague phoned me about a weird problem with enabling HA in a ESXi cluster. The following errors occurred:

  1. Configuration of host IP address is inconsistent on host : address resolved to Host misconfigured. IP address of not found on local interfaces
  2. cmd addnode failed for primary node: Internal AAM Error – agent could not start

So the first error(1.) was reported by esxhost01 and the second(2.) by esxhost02.

Let’s start with esxhost01.

So this customer had a VMotion and Management portgroup on two seperate vSwitches. This error seems to indicate that during the configuration HA is using the VMotion portgroup. These hosts have been added to VC with the management portgroup IP(IP+Name also in dns). So how do I make sure that HA isn’t using the VMotion network for HA, it’s easy go to your cluster and open up the advanced options for HA and add the following key with the value false:

  • das.allowVmotionNetworks=false

In other words, don’t use the VMotion network for the HA heartbeat. The weird thing in this case is that it shouldn’t use the VMotion network by default so there seems to be a glitch here…

So now for the second problem.

The HA(AAM) agent could not start. So just to make sure that the USB key wasn’t corrupt the key was recreated. But still this error occurred. As some of you might now, that if you want to use HA with a disk less server you will need to create a userworld swap on the SAN. (Read this KB for more info on that one…) So just to make sure that the swap wasn’t causing this problem the directory was cleaned out and and HA was reconfigured. When the directory was emptied the HA agent installed without any problem at all…

  • When reinstalling ESXi or when strange HA errors occur clean up the userworld swap!

Thanks goes out to Remco for providing me with the additional details!

HA, primary and secondary nodes?

· ·

Because I’ve been looking into HA myself I wanted to clarify things up, for you guys and for myself… writing is a good way of getting the facts straight. I’ve seen and get a lot of questions regarding HA. So I just bundled a bunch of questions I received over the last couple of months…

How does a primary and / or secondary get selected?

  • The first 5 hosts that join the VMware HA cluster are automatically selected as “primary nodes”
  • All the others are automatically selected as “secondary nodes”
  • When you do a reconfigure for HA the primary nodes and secondary nodes are selected again, this is random

What’s up with these primaries and secondaries?

  • Primary nodes hold cluster settings and all node states which are synced between primaries
  • Secondary nodes send their state info(resource occupation) to the primary nodes
  • Nodes send heartbeats to each other, primary nodes send heartbeats to primary nodes only and secondary also only to primary. And they do this every second. (Which is a changeable value: das.failuredetectioninterval)

So what if a primary node fails, will a secondary be promoted?

  • No, there will only be a new primary appointed when the failed one is removed from the cluster. A secondary will be promoted to primary at random.

But what if all my primary nodes fail?

  • This is an unaddressed issue, that’s the reason why you can only account for 4 host failures within a cluster! There needs to be at least one primary!

So when does the gateway come in play?

  • Actually the gateway, which is the default “isolation address”, will only be used when an isolation has occurred. So when the AAM client thinks it’s isolated it will check the isolation addresses.

So if anyone has a question just drop it here and I’ll try to answer it and update the above list…

VirtualCenter log files in your temp directory

· ·

By default your VirtualCenter logfiles are stored in a temp folder(as of 2.5 they are stored in: C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\Logs). VMware Wolf wrote a nice article about all the locations these log files are stuffed. For some reason I don’t get a pleasant feeling when I store my VirtualCenter (VPXD) log files in a temporary windows directory or the profile directory for that matter(thanks for the comment!!). If there’s one thing admin’s clean up first when they tend to run out of diskspace it’s their temp directory… it’s called temp for a good reason!

So in order to prevent this you could change the location of the VPXD log files very easily. Edit “vpxd.cfg”. It’s located here: %AllUsersProfile%\Application Data\VMware\VMware VirtualCenter\.

Add the following lines in the “<config>” section and change the path accordingly:

<log>
<directory>c:\VC_Logs</directory>
</log>

FT_HOSTS, where is it in ESX 3.5 U2?

· ·

This seems to be High Availability day! I was just testing my previous blog when I discovered a weird DNS or host file related error. So I opened up my console and typed “vi /etc/FT_HOSTS”. “vi” opened a blank file and reported back “new file”. What the heck, so I did a find and found that the FT_HOSTS file has been relocated to: /etc/opt/vmware/aam/. So if you’re looking for FT_HOSTS….

And for ESXi, you should be looking here “/var/run/vmware/aam/” by the way.