Yellow Bricks

by Duncan Epping

Bugs

Virtual Machines appear to be running or registered on multiple ESX Servers

· ·

I was doing my daily round on the VMTN Forums and noticed this topic on VMs flickering between ESX hosts. I’ve personally never witnessed this and didn’t even knew it was a known issue. Luckily Troy Clavell pointed the topic starter out to a KB article related to this exact issue. Apparently it’s being caused by the fact that the VM is registered on two hosts at the same time.

Symptoms:

  • After one of the following, a Virtual Machine appears as being registered on two ESX Servers:
    • A VMotion fails to complete correctly or times out in VirtualCenter
    • A DRS issue where virtual machines are VMotioned automatically in quick succession
    • When a machine is powered on during VMware HA failover.
    • The Service Console on an ESX host is low on memory starving the vpxa process
  • In VirtualCenter, you see the virtual machine as appearing on one ESX Server for a few seconds, then it seems to be on the other.
  • The virtual machine may appear to jump back and forth among different ESX hosts.

I’m not going to copy/paste the solution cause the KB article will probably change over time, but it’s most definitely worth looking into… it does sound like something that can happen to all of us.

Security updates for ESX 3.x

· ·

Just a quick note that I wanted to get out… A security patch has been released. Please look at the following KB article and download, test and implement the patch.

VMware ESX 3.5, Patch ESX350-200904201-SG: Updates VMX RPM
Issues fixed in this patch (and their relevant symptoms, if applicable) include:

  • A critical vulnerability in the virtual machine display function might allow a guest operating system to run code on the host. The Common Vulnerabilities and Exposures Project (cve.mitre.org) has assigned the name CVE-2009-1244 to this issue.

DIY Patch for VMware Workstation 6.5.1 on Linux 2.6.29 kernel

· ·

I’m running OpenSuse as my primary OS at the moment. One of the problems I had was getting VMware Workstation to work in a normal way. The problem was that the kernel modules weren’t compiled correctly. It’s been bugging me ever since, but due to the fact that it’s the end of the quarter I never had the time to actually look in to it. Luckily VMTN user Krellan did have some extra time on his hands and managed to fix the problem, Thanks!!

Fixing your broken installation is fairly easy:

To use this patch, download it from the attachment, gunzip it, then go to your /usr/lib/vmware/modules/source directory. Untar all of the files. Apply the patch with “patch -p1”. After the patch has applied, re-tar all the files. Then, run “vmware-modconfig –console –install-all” again, under Linux 2.6.29, and it should now work.

Go to the VMTN Forums and download the patch if you are having trouble getting the VMware Workstation kernel modules compiled correctly!

vCenter and SQL service dependencies

· ·

During several projects I noticed that for some reason the vCenter service would not start correctly. After a quick browse in “services.msc” and the eventlog I noticed that the vCenter service started before the SQL service. As you can imagine vCenter needs SQL to be up and running before it can actually start. I fixed it by creating a dependency. For some weird reason I never blogged this, but today I noticed this KB article that describes how to set up this dependency:

Adding a dependency to the VirtualCenter service so that it waits for SQL Express remedies this.

To create a service dependency:

  1. Click Start > Run.
  2. Type services.msc and press Enter.
  3. Locate the SQL Express instance for VirtualCenter. For example, SQL Server (SQLEXP_VIM).
  4. Open the SQL Express instance and note the Service Name. For example, MSSQL$SQLEXP_VIM .
  5. In the Run dialog, type Regedit.exe and press Enter. Browse to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\vpxd .
  6. Double-click the DependOnService key and add the Service name using the name identified in step 4.
  7. Close Regedit .
  8. Go back to the Services Panel and open the SQL Server properties.
  9. On the Dependencies tab, verify the VMware VirtualCenter service is listed as depending on the SQL service instance.

As you can see the solution is fairly easy. Keep in mind that you need to be running the SQL Server locally on the vCenter server for this to work, especially for larger environments I wouldn’t advise running both on the same box. For SMB environments this should work just fine.

vCenter tasks time-out or ESX disconnects?

· ·

I just received an email from a fellow consultant about a customer which had vCenter tasks time-out every once in a while. At times also ESX hosts got disconnected for no apparent reason at all. He discovered the following article by Richard Blythe aka VMware Wolf: ESX disconnects randomly or when doing VI client tasks from VC, task randomly timeout after a long idle time. Richard created a list of issues/errors that might be related to this issue:

  • ESX disconnects randomly from VirtualCenter
  • ESX disconnects when performing VI Client tasks from VirtualCenter.
  • Tasks randomly timeout after a long idle time
  • “An error occurred communicating to the remote host” pops up.

The article refers to an issue with vCenter Update 3 in combination with firewalls using state-ful inspection. The problem occurs because of SOAP timeouts, and this behavior did not exist in VC 2.0.x or 2.5 GA, as they used a different mechanism to communicate with ESX. The official KB article hasn’t been released yet but a temporary workaround has been published by Richard. If you run into any of the before mentioned issues head over to Richard’s website and try out the workaround until the fix or official KB article is released.