Yellow Bricks

by Duncan Epping

stretched cluster

vSAN ESA Witness memory and CPU resources?

· · Leave a Comment

Not sure when this happened, but somehow the resource requirements for the vSAN Witness VM disappeared. Someone asked me last week how much memory is allocated to a VM, and how many vCPUs. Now, of course, this depends on the profile you select as the Witness VM has an M, L, and XL profile. The profile you select is determined by the number of VMs you will be provisioning, yes it is smart to take a growth factor into account. Now, when you deploy the VM, it doesn’t give a hint either, but you can figure out the size by simply looking at the OVF descriptor file. So this is what I got from the vSAN ESA Witness OVF:

  • vSAN ESA Witness XL – 8 vCPUs – 64 GB memory
  • vSAN ESA Witness L – 4 vCPUs – 32 GB memory
  • vSAN ESA Witness M – 4 vCPUs – 16 GB memory

And for those who were wondering, with vSAN OSA the requirements are:

  • vSAN OSA Witness XL – 6 vCPUs – 32 GB memory
  • vSAN OSA Witness L – 2 vCPUs – 32 GB memory
  • vSAN OSA Witness Normal – 2 vCPUs – 16 GB memory
  • vSAN OSA Witness Tiny – 2 vCPUs – 8 GB memory

I hope that helps, and also please do note… if you read this article a few years from now, things may have changed!

Can I replicate, or snapshot, my vSAN Stretched Cluster Witness appliance for fast recovery?

· · Leave a Comment

I’ve been seeing this question pop up more frequently, can I replicate or snapshot my vSAN Stretched Cluster Witness appliance for fast recovery? Usually, people ask this question as they cannot adhere to the 3-site requirement for a vSAN Stretched Cluster. So by setting up some kind of replication mechanism with low RPO, they try to mitigate this risk.

I guess the question stems from a lack of understanding of what the witness does. The witness provides a quorum mechanism, the quorum mechanism helps determine which site has access to the data in the case of a network failure (ISL) between the data locations.

Can I replicate, or snapshot, my vSAN Stretched Cluster Witness appliance for fast recovery?

So why can the Witness Appliance not be snapshotted or replicated then? Well, in order to provide this quorum mechanism, the Witness Appliance stores a witness component for each object. This is not per site, or per VM, but for every object… So if you have a VM with multiple VMDKs, you will have multiple witness objects per VM stored on the witness appliance. That witness object holds metadata and, through a log sequence number, understands which object holds the most recent data. This is where the issue arises. If you revert a Witness Appliance to an earlier point in time, the witness components also revert to an earlier point in time, and will have a different log sequence number than expected. This results in vSAN being unable to make the object available to the surviving site, or the site that is expected to hold quorum.

So in short, should you replicate or snapshot the Witness Appliance? No!

 

vSAN 9.0 Site Maintenance Mode demo!

· · Leave a Comment

I had a few questions about this, so I figured I would record a quick demo showing Site Maintenance. In the demo, I have a stretched cluster configured in vSAN 9.0, and I am going to place the Preferred Site into maintenance mode. First a pre-check will occur to verify all workloads are replicated between locations, and then the site is placed into maintenance while maintaining data consistency across hosts. Next demo I will record will show the Manual Site Takeover command that was also introduced in 9.0 for OSA, but will be also available soon for ESA.

What do I do after a vSAN Stretched Cluster Site Takeover?

· · 4 Comments

Over the last couple of months, various new vSAN features were announced. Two of those features are around the Stretched Cluster configuration, and have probably been the number 1 feature request for a few years. Now that we have Site Takeover and Site Maintenance functionality available, I am starting to get some questions about the impact of them, and in particular, the Site Takeover functionality is raising some questions.

For those who don’t know what these features are, let me describe them briefly:

Site Maintenance = The ability to place a full vSAN stretched cluster Fault Domain into maintenance mode at once. This ensures that all hosts within the fault domain have consistently stored the data, and all hosts will go into maintenance mode at the same time.

Site Takeover = This provides the ability when a Witness and a Data Site has failed to bring back the remaining site through a command line interface. This will reconstruct the remaining “site local” RAID configuration, making the objects available again, which will then allow vSphere HA to restart the VMs.

Now, the question that the above typically raises is what happens to the Witness and the Data Site that failed when you do the Site Takeover? If you look at the VMs RAID configuration, you will notice that both the Witness and the Data Site components of the sites that failed will completely disappear from the RAID configuration.

Can I replicate, or snapshot, my vSAN Stretched Cluster Witness appliance for fast recovery?But what do you do next, because even after you run the Site Takeover, you still see your hosts and the witness in vCenter Server, and you still see a stretched cluster configuration in the UI. Now at first I thought that if the environment was completely up and running again, you had to go through some manual effort to reconstruct the stretched cluster. Basically, remove the failed hosts, wipe the disks, and recreate the stretched cluster. This is, however, not the case.

In the example above, if the Preferred site and the Witness site return for duty, vSAN will automatically discard the stale components in those previously failed sites. It will recreate new components for all objects, and it will do a full resync of the data.

If you end up in a situation where your hosts are completely gone (let’s say as a result of a fire), then you will have to do some kind of manual cleanup as follows, before you rebuild and add hosts back:

  • Remove the failed hosts from the vCenter inventory
  • Remove the witness from the vCenter inventory
    • Delete the witness from the vCenter Server it is running, a real delete!
  • Delete the surviving Fault Domain, this should be the only Fault Domain still listed in the vCenter interface
  • You now have a normal cluster again
  • Rebuild hosts and recreate the stretched cluster

I hope that helps,