I was just reading some of the comments posted today and Marc Sevigny, one of the vSphere HA developers, pointed out something which I did not know. I figured this is probably something that many are not aware of so I copied and pasted his comment:
Another thing to check if you experience this error is to see if you have jumbo frames enabled on the management network, since this interferes with HA communication.
This is document here in a note: http://kb.vmware.com/kb/2006729
To make it crystal clear: disable jumbo frames on your management network with vSphere 5.0 as there’s a problem with it! This problem is currently being investigated by the HA engineering team and will hopefully be resolved.
<Update> Just received an email that all the cases where we thought vSphere HA issues were caused by Jumbo Frames being enabled were actually caused by the fact that it was not configured correctly end-to-end. Please validate Jumbo Frame configuration on all levels when configuring. (Physical Switches, vSwitch, Portgroup, VMkernel etc)</Update>
The KB link in this article goes to a Page Not Found @VMware.
In order to enable Jumbo Frames I enable at the virtual switch level and the port group level. Is it ok to enable Jumbo Frames at the virtual switch level and not enable Jumbo Frames only on the management vmkernel port group? I’d still like to have it enabled on other port groups.
This definitely isn’t good. I hope to see this remedied soon.
The correct URL for the KB article is http://kb.vmware.com/kb/2006729
Jas
though the article makes sense, I’m still confused 🙁 How do you disable Jumbo frames for a port group that uses active/active connections. Or even so, active/standby. Other than having your Management network on it’s on vSwith, I don’t see how this is possible when you have the vMotion and Management network on the same vSwitch using 2 pNICs set to active/active. Any clarification would be greatly appreciated.
I see now, only relevant to vSphere5. 🙂
Jumbo Frames are enabled at the virtual switch AND individual port group(s). So, you should be able to edit the properties of your management vmkernel and set the MTU to a default 1500.
One other question is raised here in terms of jumbo frames on virtual interfaces that are delivered to blade servers. (i.e. HP or Cisco). Does the entire path need to have jumbo’s turned off? I would agree that simply setting the vSwitch to 1500 should suffice, but node to node communication will go through virtual connect or the UCS switches.
To disable jumbo frames on the vmk port that has mgmt traffic enabled just select the vmk and edit the properties and change the MTU to 1500. No change to the vSwitch or network required as a result.
On the HP VirtualConnect case, you don’t need to configure anything, the Jumbo Frames are enabled by default. There’s no way to change that configuration.
Jumbo frames are an end to end configuration. Provided your vmk interface for management is set to 1500 mtu it doesn’t matter if the vswitch or physical switch are higher as the management interface will only send packets up to its configured 1500 MTU.
so jumbo frames being enabled everywhere else is ok?
Yes it is.
It looks like both of the KB links above are incorrect. The correct URL for this issue is:
http://kb.vmware.com/kb/2011974
Btw. I’ve configured Jumbo Frames on my management network (vSphere 5) and haven’t had a problem with HA.