Yellow Bricks

by Duncan Epping

Archives for 2009

The Basics: How to kill a VM that’s stuck during shutdown?

· ·

I just replied to a topic on the VMTN forums and thought it might be useful to write it down here as well. (I thought I already did, but a search didn’t turn up anything.)

When a VM gets stuck during shutdown or isn’t responding anymore you can easily kill the VM. First option is the command line version of vCenter’s “shutdown vm”, first list all the VMs running on the host so you can copy and paste the <config> in to the next command. The command “vmware-cmd <config> stop trysoft” will try to initiate a soft shutdown first, in other words a shutdown via the Guest OS, if that doesn’t work it will do a power off. Now, as most of you probably already experienced, sometimes it’s impossible to shutdown the VMs in a normal way. This is where 2nd, 3rd and 4th option come in to place. Option two uses vm-support to kill the VM, use “-x” to list the VM id’s and kill it with “-X”. The third option uses vimsh, in this case we use vmware-vim-cmd, “vmsvc/getallvms” lists all VMs and the id’s and with “vmsvc/poweroff” you can specify the VM that needs to be powered off. The fourth option is the Linux/Unix way of doing it, find the process id of the VM via “ps -auxwww” and just kill it.

  1. vmware-cmd -l
    vmware-cmd <config> stop trysoft
  2. vm-support -x
    vm-support -X <vmid>
  3. vmware-vim-cmd vmsvc/getallvms
    vmware-vim-cmd vmsvc/poweroff <vm id>
  4. ps -auxwww | grep <vm name>
    kill <process id>
  5. if option three isn’t successful do the following: 
    kill -9 <process id>

As VMwareWolf points out, there’s an excellent KB article on this subject to be found here: http://kb.vmware.com/kb/1004340

SRM install log files

· ·

I was rebuilding my lab this evening and ran into some weird problem during the installation. I wanted to debug it but couldn’t find the log files. After a quick search on the VMTN forum I found the following, which creates a log file called “SRMinstall-log.txt”:

VMware-srm-1.0.1-128004.exe /V"/lve SRMinstall-log.txt"

I know it’s a weird looking format, but it actualy works! Of course if you’re using a different version you should change the .exe accordingly. Thanks Lee aka Smoggy.

EMC announced the Symmetrix V-Max!

· ·

EMC has just announced the Symmetrix V-Max. Here are the specs, what a beast:

The first new Symmetrix model based on the Virtual Matrix Architecture is the Symmetrix V-Max storage system, the world’s largest high-end storage array, featuring:

  • Up to 128 Intel Xeon processor cores
  • Up to 1 TB (terabyte) of global memory
  • Fibre Channel/FICON/Gigabit Ethernet/iSCSI connectivity
  • Latest generation Flash/Fibre Channel/SATA drive support
  • Scale to 2,400 drives
  • Maximum usable, protected capacity of 2 PBs (petabytes)

Of course, as the name suggests, the Symmetrix has been optimized for Virtualized Datacenters. Here’s just one example of how the Symmetrix V-Max will make the life of storage admins a lot easier:

Re-architecting the way that Enginuity interacts with the host OS layers, especially in the virtualization space. Creating the ability to dynamically provision entire port groups, initiator groups, even topologies with less steps than it takes to create a MetaLUN in Navisphere. I’m sure that Chad Sakac will have more to say regarding this but, let’s be REALLY clear on this: putting a Symmetrix V-Max into your virtualized environment is now going to be even easier than…well, most other things out there. We’re calling this Auto-provisioning, by the way, and to put it in the words of wizened genius within the walls of EMC (Duane Olson, if I may be so bold):

What a concept. Create a group for a particular host(ESX farm as an example), and now all you do is create/assign storage to this group and magically, the host has new storage.. One command, and its done. No more create the device, assign a device to a frontend channel, and then mask the device to a host…One step, and its all done. This will significantly decrease the time needed to allocate new storage to an existing host/hosts.

And for those thinking about BC-DR:

EMC is introducing the new zero-data-loss SRDF Extended Distance Protection (EDP) feature for Symmetrix V-Max systems, which can reduce the cost of multi-site replication by up to 50 percent. SRDF is ideal for virtual server environments and has been integrated with VMware Site Recovery Manager (SRM) and supports EMC Replication Manager for automated protection of VMware environments.

For more info check out these three blog articles:

Security updates for ESX 3.x

· ·

Just a quick note that I wanted to get out… A security patch has been released. Please look at the following KB article and download, test and implement the patch.

VMware ESX 3.5, Patch ESX350-200904201-SG: Updates VMX RPM
Issues fixed in this patch (and their relevant symptoms, if applicable) include:

  • A critical vulnerability in the virtual machine display function might allow a guest operating system to run code on the host. The Common Vulnerabilities and Exposures Project (cve.mitre.org) has assigned the name CVE-2009-1244 to this issue.

Ken’s view on Service Console redundancy and my take…

· ·

I’ve read Ken’s article on Service Console redundancy a couple of times, When is it OK to default on your VI? As I also wrote on the VMTN Blog I really love Ken’s posts so far. They are in depth and Ken knows what he is talking about. His argument, keep it simple, make sense.

Basically, what we’ve done is to let everything default. All the adapters are active, the load balancing method is virtual switch port based and nothing is overridden by the port groups.

But I actually don’t agree with Ken on this one. I never use “virtual port id” load balancing for the Service Console and VMkernel, especially not if I combine these two port groups on one vSwitch.

Call me a control freak if you like, but I want to know which port group is using which vmnic. I always use an Active/Standby scenario for the vSwitch that holds the Service Console and VMkernel. Let me steal Ken’s excellent diagram to give you an idea what I’m talking about:

If anything goes wrong there’s full redundancy, which is a must have. The setup can be scripted in a couple of lines and if you need to troubleshoot you know exactly which physical NIC is being used for what purpose. The Service Console and the VMkernel/VMotion are just too important to be guessing where they are running in my opinion, especially in large environments. I want every server to be exactly the same, I don’t want to have the Service Console running on vmnic0 on the first server and on vmnic2 on the next. Like I said… I like to be in control, full control.

For those who want to set this up via a scripted install:

/usr/bin/vimsh -n -e “hostsvc/net/portgroup_set
–nicorderpolicy-active=vmnic0 –nicorderpolicy-standby=vmnic2 vSwitch0
‘Service Console’”
/usr/bin/vimsh -n -e “hostsvc/net/portgroup_set
–nicorderpolicy-active=vmnic2 –nicorderpolicy-standby=vmnico vSwitch0 VMkernel”