Yellow Bricks

by Duncan Epping

8.0

Are the vSAN disks encrypted or not, and is the environment health?

· · Leave a Comment

There was an internal question that came up, and I figured I would write a quick article as I had to grab some screenshots anyway. If you have vSAN Encryption – Data At Rest enabled, how do you verify the disks are actually encrypted? There are a couple of things you can do, and one is, of course verify in the vSAN UI that encryption is enabled in the configuration section. But you can also verify on a per-host basis if the disks have been encrypted through the command: esxcli vsan storage list. The output would look as follows:

Are the vSAN disks encrypted or not, and is the environment health?

As you can see, Encryption: true.

Of course, it is also beneficial to know if the Key Management System is reachable and healthy, as well as whether the necessary CPU instructions are available. These details can be viewed in vSAN Skyline Health, as shown in the next screenshot.

Hope that helps… OH, if you do use the Native Key Server, and encounter an error “not available on host”, verify if you enabled it with “Use key provider only with TPM” ticked or not, as if that is selected and you don’t have a TPM would result in that error.

Doing network/ISL maintenance in a vSAN stretched cluster configuration!

· ·

I got a question earlier about the maintenance of an ISL in a vSAN Stretched Cluster configuration which had me thinking for a while. The question was what would you do with your workload during maintenance. I guess the easiest of course is to power off all VMs and simply shutdown the cluster, for which vSAN has a UI option, and there’s a KB you can follow. Now, of course, there could also be a situation where the VMs need to remain running. But how does this work when you end up losing the connection between all three locations? Normally this would lead to a situation where all VMs will become “inaccessible” as you will end up losing quorum.

As said, this had me thinking, you could take advantage of the “vSAN Witness Resiliency” mechanism which was introduced in vSAN 7.0 U3. How would this work?

Well, it is actually pretty straight forward, if all hosts of 1 site are in maintenance mode, failed, or powered off, the votes of the witness object for each VM/Object will be recalculated within 3 minutes. When this recalculation has completed the witness can go down without having any impact on the VM. We introduced this capability to increase resiliency in a double-failure scenario, but we can (ab)use this functionality also during maintenance. Of course I had to test this, so the first step I took was placing all hosts in 1 location into maintenance mode (no data evac). This resulted in all my VMs being vMotioned to the other site.

Now next I checked with RVC if my votes were recalculated or not. As stated, depending on the number of VMs this can take around 3 minutes in total, but usually will probably be quicker. After the recalculation had been completed I powered off the Witness, and this was the result as shown below, all VMs were still running.

Of course, I had to double check on the commandline using RVC (you can use the command “vsan.vm_object_info” to check a particular object for instance) to ensure that indeed the components of those VMs were still “ACTIVE” instead of “ABSENT”, and there you go!

Now when maintenance has been completed, you simply do the reverse, you power on the witness, and then you power on the hosts in the other location. After the “resync” has been completed the VMs will be rebalanced again by DRS. Note, DRS rebalancing (or should rules being applied) will only happen when the resync of the VM has been completed.

vSphere 8.0 U2 and vSAN 8.0 U2 just shipped, learn all about it here!

· ·

vSphere 8.0 U2 and vSAN 8.0 U2 just shipped, and of course the Unexplored Territory Podcast has already covered this. If you want to learn all about it make sure to listen to the episode below. Or of course read the release notes (vCenter, ESXi, vSAN).

You can find the vSAN 8.0 U2 episode on Spotify (https://bit.ly/3QNjpFk), and Apple (https://bit.ly/3QPt7XL), as well as any other podcast app, or simply listed via the embedded player below!

You can find the vSphere 8.0 U2 episode on Spotify (https://bit.ly/3snOh5l), Apple (https://bit.ly/45lRK2Q), as well as any other podcast app, or simply listed via the embedded player below!

Deleting the vCLS VMs using Retreat Mode starting with vSphere 8.0 U2

· ·

I posted about “retreat mode” and how to delete the vCLS VMs when needed a while back, including a quick demo. Back then you needed to configure an advanced setting for a cluster if you wanted to delete the VMs for whatever reason. (Usually for troubleshooting purposes people would do a delete/recreate.) Starting with vSphere 8.0 U2 you can now use the UI to enable retreat mode on a per cluster level. How do you do this? well fairly straight forward:

  • Click on the cluster you would want to delete the VMs for
  • Click on Configure
  • Click on “General” under “vSphere Cluster Services”
  • Click on “EDIT VCLS MODE”
  • Click on “Retreat Mode” and click “OK”

Now the VMs will be deleted, if you want to recreate the VMs, follow the same procedure, but change “Retreat Mode” to “System Managed”. I tested the process yesterday and created a quick demo for you:

Scalable Snapshots demo with the vSAN 8.0 Express Storage Architecture

· ·

Starting with vSAN 8 a brand new architecture was introduced called “Express Storage Architecture”. Over the last year or so a lot of information has been shared about ESA and the benefits of ESA. One of the things which ESA introduces is much-improved snapshot scalability.

With vSAN OSA, and with VMFS, when you create a snapshot you typically immediately see a performance degradation. This is because both VMFS and vSAN OSA still operate using the redo-log based snapshot mechanism. This means that with vSAN OSA when you create a snapshot a new object is created and writes are re-directed. It also means that reads will be coming from various files, if you have one or more snapshots. This mechanism is, unfortunately, not very effective. Let me borrow a diagram that is part of a post John Nicholson wrote to demonstrate that old logic.

With vSAN 8 ESA the mechanism has changed and no longer does vSAN, or vSphere for that matter, create an additional object. vSAN ESA handles this on a meta-data level. In other words, instead of redirecting writes and traversing files for reads, vSAN now leverages a highly efficient B-Tree structure and pointers to keep track of which block is associated with which snapshot.

Not only is this more efficient from a capacity perspective, but more importantly it is very efficient from a performance standpoint. I ran half a dozen tests in my lab, and what I saw was a below 2% performance impact between a VM without a snapshot and a VM with one or multiple snapshots. I could NOT see a significant difference between the first or the fifth snapshot. I do want to point out that my lab is not officially certified to run vSAN ESA, nevertheless, I was very impressed with the results.

During the last run, I actually recorded the whole exercise. In this demo, I show the creation of one snapshot, while the VM is running a benchmark (HCIBench). Now, during the testing, I created not one but various snapshots and of course, I deleted all of them as well. You have all probably experienced extensive stun times during the deletion of a snapshot at times, and this is where vSAN ESA shines. The stun times have been reduced by 100 times, and that is something I am sure each of you will appreciate. Why have they been reduced drastically? Well, simply because we no longer have to copy data from one vSAN object to another. This makes a huge difference, not just for stun times, but also for performance in general (latency, IOPS, throughput). If you are interested, have a look at the demo!