Yellow Bricks

by Duncan Epping

u1

VMware Virtual SAN launch and book pre-announcement!

· ·

Today is the day, finally… the Virtual SAN (VSAN) launch. Many people have been waiting for this one. With 12.000 plus beta participants this was one of the biggest projects I have ever seen within VMware. It is truly impressive to see how the product has grown and what the team has done. Before I will provide you with some of the details of the announcement I want to share something else that all of you should look out for:

Cormac Hogan and I decided it was time for a book on Virtual SAN. Both of us have published many articles about VSAN the last 9 months and have been working with the product for over a year now so it only made sense. We have decided, and this wasn’t an easy decision for me, to go with VMware Press. When I say “not an easy decision” I don’t want to sound negative about using publisher, but it is just that I have had a great experience (and results) with self-publishing. It was time for a new experience though, try something different. As we speak we are working hard to get the final set of chapters in for review / editing and we are hoping to have the book available before VMworld. I am guessing that the rough cuts will be available through Safari in the upcoming weeks, if so I will let you know via a blog post.

Now lets get back to the topic of the day again, Virtual SAN Launch… So what was announced today?

  • General Availability of Virtual SAN 1.0 the week of the 10th of March
  • vSphere 5.5 Update 1 will support VSAN GA
  • Support for 32 hosts in a Virtual SAN cluster
  • Support for 3200 VMs in a Virtual SAN cluster
  • Full support for VMware Horizon / View
  • Elastic and Linear Scalability for both capacity and performance
  • VSAN is not a VSA. Performance is much better than any VSA!
  • 2 Million IOPS validated in a 32 host Virtual SAN cluster
  • ~ 4.5PB in a 32 host cluster
  • 13 different VSAN Ready Node configurations between Cisco IBM Fujitsu and Dell available at GA, more coming soon!

Once again, great work by the VSAN team. Version 1.0 just got release, and I can barely wait for the next release to become available!

vSphere HA advanced settings, the KB

· ·

I’ve posted about vSphere HA advanced settings various times in the past, and let me start by saying that you shouldn’t play around with them unless you have a requirement to do so. But if you do, there is a KB article which I can highly recommend as it lists all the known and lesser known advanced settings. I had the KB article updated with vSphere 5.5 advanced settings yesterday (Thanks KB team for being so responsive!) but it also applies to vSphere 5.0 and 5.1. Recommended read for those who want to get in to the nitty gritty details of vSphere HA.

http://kb.vmware.com/kb/2033250

Unmounting datastore fails due to vSphere HA?

· ·

On the VMware Community Forums someone reported he was having issues unmounting datastores when vSphere HA was enabled. Internally I contacted various folks to see what was going on. The error that this customer was hitting was the following:

The vSphere HA agent on host '<hostname>' failed to quiesce file activity on datastore '/vmfs/volumes/<volume id>'

After some emails back and forth with Support and Engineering (awesome to work with such a team by the way!) the issue was discovered and it seems that in two separate instances issues were resolved that had to do with unmounting of datastores. Keith Farkas explained on the forums how you can figure out if you are hitting those exact problems or not and in which release they are fixed, but at I realize those kind of threads are difficult to find I figured I would post it here for future reference:

You can determine if you are encountering this issue by searching the VC log files. Find the task corresponding to the unmount request, and see if the follow error message is logged during the task’s execution (Fixed in 5.1 U1a) :

2012-09-28T11:24:08.707Z [7F7728EC5700 error 'DAS'] [VpxdDas::SetDatastoreDisabledForHACallback] Failed to disable datastore /vmfs/volumes/505dc9ea-2f199983-764a-001b7858bddc on host [vim.HostSystem:host-30,10.112.28.11]: N3Csi5Fault16NotAuthenticated9ExceptionE(csi.fault.NotAuthenticated)

While we are on the subject, I’ll also mention that there is another know issue in VC 5.0 that was fixed in VC5.0U1 (the fix is in VC 5.1 too). This issue related to unmounting a force mounted VMFS datastore. You can determine whether you are hitting this error by again checking the VC log files. If you see an error message such as the following with VC 5.0, then you may be hitting this problem. A work around, like above, is to disable HA while you unmount the datastore.

2011-11-29T07:20:17.108-08:00 [04528 info 'Default' opID=19B77743-00000A40] [VpxLRO] -- ERROR task-396 -- host-384 -- vim.host.StorageSystem.unmountForceMountedVmfsVolume: vim.fault.PlatformConfigFault:

vSphere 5.0 U1a was just released, vDS/SvMotion bug fixed!

· ·

Many of you who hit the SvMotion / VDS / HA problem requested the hotpatch that was available for it. Now that Update 1a has been released with a permanent fix how do you go about installing it? This is the recommended procedure:

  1. Backup your vCenter Database
  2. Uninstall the vCenter hot-patch
  3. Install the new version by pointing it to the database

The reason for this is that the hot-patch increased the build number, and this could possibly conflict with later versions.

And for those who have been waiting on it, the vCenter Appliance has also been update to Update 1 and now includes a vPostgress database by default instead of DB2!

Answering some admission control questions

· ·

I received a bunch of questions on HA admission control in this blog post and I figured I would answer them in a blog post so that everyone would be able to find / read it. This was the original set of questions:

There are 4 ESXi Hosts in the network and 4 VMs (Same CPU, RAM Reservation for all VMs) on each Host. Admission Control is policy is set to ‘Host failure cluster tolerates’ to 1. All the available 12 slots have been used by the powered ON VMs, except the 4 reserved slots for failover.
1) What happens if 2 ESXi Hosts fails now? ( 2 * 4 VMs needs to fail over). Will HA restart only 4 VMs as it has only 4 slots available? And Restart of the remaining 4 VM fails?
Same Scenario, but Policy is set to ‘% of cluster resources reserved’ = 25%. All the available 75 % resources have been utilized by all the 16 VMs, except 25 % reserved for failover
2) What happens if 2 ESXi Hosts fails now? ( 2 * 4 VMs needs to fail over). Will HA restart only 4 VMs as it consumes 25 % of resources? And Restart of the other 4 VM fails?
3) Does HA check the VM reservation (or any other factor) at the time of restart ?
4) HA only restart a VM if the Host could guarantee the reserved resources or restart Fails?
5) What if no VM reservations are set VM level ?
6)What does HA takes into consideration when it has to restart VMs which has no reservation ?
7)Will it guarantee the configured Resources for each VMs ?
8)If not, How HA can restart 8 VMs (as per our eg) when it only has configured reserved resources for just 4 VM
9)Will it share the reserved resources across 8 VMs and will not care about the resource crunch or is it about first come first serve
10)Admission control doesn’t have any role at all in the event of HA failover ?

Let me tackle these questions one by one:

  1. In this scenario 4 VMs will be restarted and 4 VMs might be restarted! Note that the “slot size” policy is used and that this is based on the worst case scenario. So if your slot is 1GB and 2GHz but your VMs require way less than that to power-on it could be all VMs are restarted. However, HA guarantees the restart of 4 VMs. Keep in mind that this scenario doesn’t happen too often, as you would be overcommitting to the extreme here. As said HA will restart all VMs it can. It just needs to be able to satisfy the resource reservations on memory and CPU!
  2. Again, also in this HA will do its best to restart. It can restart new VMs until all “unreserved capacity” is used. As HA only needs to guarantee reserved resources chances of hitting this is very slim, as most people don’t use reservations at a VM level it would mean you are overcommiting extremely
  3. Yes it will validate if there is a host which can back the the resource reservations before it tries the restart
  4. Yes it will only restart the VM when this can be guaranteed. If it cannot be then HA can call,”DRS” to defragment resources for this VM
  5. If there are no reservations then HA will only look at the “memory overhead” in order to place this VM
  6. HA ensures the portgroup and datastore are available on the host.
  7. It will not guarantee configured resources, HA is about restarting virtual machines not about resource management. DRS is about resource management and guaranteeing access to resources.
  8. HA will only be able to restart the VM if there are unreserved resources available to satisfy the VMs request
  9. All resources required for a virtual machine need to be available on a single host! Yes resources will be shared on a single host, just as long as no reservations are defined.
  10. No Admission Control doesn’t have any role in an HA failover. Admission Control happens on a vCenter level, HA failovers happen on an ESX(i) level.