I’ve been seeing this question pop up more frequently, can I replicate or snapshot my vSAN Stretched Cluster Witness appliance for fast recovery? Usually, people ask this question as they cannot adhere to the 3-site requirement for a vSAN Stretched Cluster. So by setting up some kind of replication mechanism with low RPO, they try to mitigate this risk.
I guess the question stems from a lack of understanding of what the witness does. The witness provides a quorum mechanism, the quorum mechanism helps determine which site has access to the data in the case of a network failure (ISL) between the data locations.
So why can the Witness Appliance not be snapshotted or replicated then? Well, in order to provide this quorum mechanism, the Witness Appliance stores a witness component for each object. This is not per site, or per VM, but for every object… So if you have a VM with multiple VMDKs, you will have multiple witness objects per VM stored on the witness appliance. That witness object holds metadata and, through a log sequence number, understands which object holds the most recent data. This is where the issue arises. If you revert a Witness Appliance to an earlier point in time, the witness components also revert to an earlier point in time, and will have a different log sequence number than expected. This results in vSAN being unable to make the object available to the surviving site, or the site that is expected to hold quorum.
So in short, should you replicate or snapshot the Witness Appliance? No!
