Yellow Bricks

by Duncan Epping

Server

Storage Filters

· ·

I was reading about Storage Filters last week and wanted to do a short write up. I totally forgot about it until I noticed this new KB article. The KB article only discusses the LUN filters though and not the other filters that are available today.

Currently 4 filters have been made public:

  1. config.vpxd.filter.hostRescanFilter
  2. config.vpxd.filter.vmfsFilter
  3. config.vpxd.filter.rdmFilter
  4. config.vpxd.filter.SameHostAndTransportsFilter

The first filter on the list is one I discussed roughly a year ago. The “Host Rescan Filter” makes it possible to disable the automatic storage rescan that occurs on all hosts after a VMFS volume has been created. The reason you might want to avoid this is when you adding multiple volumes and want to avoid multiple rescans but just initiate a single rescan after you create your final volume. By setting “config.vpxd.filter.hostRescanFilter” to false the automatic rescan is disabled. In short the steps needed:

  1. Open up the vSphere Client
  2. Go to Administration -> vCenter Server
  3. Go to Settings -> Advanced Settings
  4. If the key “config.vpxd.filter.hostRescanFilter” is not available add it and set it to false

To be honest this is the only storage filter I would personally recommend using. For instance “config.vpxd.filter.rdmFilter” when set to “false” will enable you to add a LUN as an RDM to a VM while this LUN is already used as an RDM by a different VM. Now that can be useful in very specific situations like when MSCS is used, but in general should be avoided as data could be corrupted when the wrong LUN is selected.

The filter “config.vpxd.filter.vmfsFilter” can be compared to the RDM filter as when set to false it would enable you to overwrite a VMFS volume with VMFS or re-use as an RDM. Again, not something I would recommend enabling as it could lead to loss of data which has a serious impact on any organization.

Same goes for “config.vpxd.filter.SameHostAndTransportsFilter”. When it is set to “False” you can actually add an “incompatible LUN” as an extend to an existing volume. An example of an incompatible LUN would for instance be a LUN which is not presented to all hosts that have access to the VMFS volume it will be added to. I can’t really think of a single reason to change the defaults on this setting to be honest besides troubleshooting, but it is good to know they are there.

Most of the storage filters have its specific use cases. In general storage filters should be avoided, except for “config.vpxd.filter.hostRescanFilter” which has proven to be useful in specific situations.

Standby vCenter Server for disaster recovery

· ·

I was reading through some documentation and found a piece on creating a cold Standby vCenter server. This used to be a common practice with vCenter 2.5 and it worked well as vCenter itself was more or less stateless.

With vSphere 4.0 something changed. Although at first it might not seem substantial it actually is. As of vSphere 4.0 VMware started using ADAM. ADAM is most commonly referred to as the component which enables Linked Mode. Linked Mode gives you the opportunity to manage multiple vCenter Server from a single pane of glass.

Not only will you have a single pane of glass you will also have a central store for roles and permissions. This is key! Roles and permissions are stored in ADAM.

Lets assume you have just a single vCenter Server and are not using Linked Mode. This will not impact the way vCenter Server stores its roles and permissions… it will still use ADAM. Even when cloned daily full consistency can not be guaranteed and as such I would personally not recommend using a cold Standby vCenter Server unless you are willing to take the risks and have fully tested it.

Standby NICs in an “IP-Hash” configuration

· ·

I was reviewing a document today and noticed something that I’ve seen a couple of times already. I already wrote about Active/Standby set ups for etherchannels a year ago but this is a slight different variant. Frank also wrote a more extensive article on it a while and I just want to re-stress this.

Scenario:

  • Two NICs
  • 1 Etherchannel of 2 links
  • Both Management and VMkernel traffic on the same switch

I created a simple diagram to depict this:

nics

In the above scenario each “portgroup” is configured in an active/standby scenario. So let’s take the Service Console. It has VMNIC0 as active and VMNIC1 as standby. The physical switch however is configured with both NICs active in a single channel.

Based on the algorithm that etherchannels use either of the two VMNICs will accept inbound traffic. The Service Console however will only send traffic outbound via VMNIC0. Even worse, the Service Console isn’t actively listening to VMNIC1 for incoming traffic as it was placed in Standby mode. Standby mode means that it will only be used when VMNIC0 fails. In other words your physical switch will think it can use VMNIC1 for you Service Console but your Service Console will not see the traffic coming in on VMNIC1 as it is configured in Standby mode on the vSwitch. Or to quote from Frank’s article…

it will sit in a corner, lonely and depressed, wondering why nobody calls it anymore.

High physical switch CPU load?

· ·

One of my customers experienced high CPU load on their physical switch. After some investigation they noticed broadcasts packets being sent every two seconds. The first reaction was Beacon Probing is probably enabled.

Unfortunately this wasn’t the case. But VMware GSS came to the rescue and pointed us towards a KB article. Apparently a bug has been identified in 4.0 which causes this behaviour:

src: http://kb.vmware.com/kb/1024435

Problem:

  • ESX sends Beacon Packets when vDS/vSwitch are connected to more than one uplink.
  • ESX server sends periodic broadcast of Beacon Packets even if the vSwitch/vNetwork Distributed Switch (vDS) is not configured to use Beacon Probing for Network Failover Detection.
  • These packets have the virtual MAC of the vmnic in the Source MAC Address field.

Workaround:

#esxcfg-advcfg -s 0 /Net/MaxBeaconsAtOnce

The customer implemented this workaround and the problem is gone… From what I have been told this issue does not exist in ESX(i) 4.1 so if you are experiencing it, an upgrade might be a better solution. In this case due to the size of the environment that was not an option.

HA Cli

· ·

I was just playing around with the HA Cli and noticed that when you give an “ln” (listNodes) that the failover coordinator (aka master primary) is also listed. I have never noticed this before, but don’t have a pre-vSphere 4.1 environment to test it on to see if this existed before 4.1. If you want to test it in your own environment just simply run “/opt/vmware/aam/bin/Cli” and give the “ln” command as shown in the screenshot below:

I also tested demoting of a node just for fun. In this case I demoted the node “esxi1” from primary to secondary:

And of course I promoted it again to primary:

 

** Disclaimer: This article contains references to the words master and/or slave. I recognize these as exclusionary words. The words are used in this article for consistency because it’s currently the words that appear in the software, in the UI, and in the log files. When the software is updated to remove the words, this article will be updated to be in alignment. **