Yellow Bricks

by Duncan Epping

vcls

vSphere 7.0 U3 contains two great vCLS enhancements

· ·

I have written about vCLS a few times, so I am not going to explain what it is or what it does (detailed blog here). I do want to talk about what is part of vSphere 7.0 U3 specifically though as I feel these features are probably what most folks have been waiting for. Starting with vSphere 7.0 U3 it is now possible to configure the following for vCLS VMs:

  1. Preferred Datastores for vCLS VMs
  2. Anti-Affinity for vCLS VMs with specific other VMs

I created a quick demo for those who prefer to watch videos to learn these things if you don’t skip to the text below. Oh and before I forget, a bonus enhancement is that the vCLS VMs now have a unique name, this was a very common request from customers.

Why would you need the above functionality? Let’s begin with the “preferred datastore” feature, this allows you to specify where the vCLS VMs need to be provisioned to from a storage point of view. This would be useful in a scenario where you have a number of datastores that you would prefer to avoid. Examples could be datastores that are replicated or a datastore that is only intended to be used for ISOs and templates, or maybe you prefer to provision on hybrid storage versus flash storage.

So how do you fix this? Well, it is simple, you click on your cluster object. You then click on “Configure”, and on “Datastores” under “vSphere Cluster Services”. Now you will see “VCLS Allowed”, if you click on “ADD” you now will be able to select the datastores to which these vCLS VMs should be provisioned.

vSphere 7.0 U3 contains two great vCLS enhancements

Next up Anti-Affinity for vCLS. You would this feature for situations where for instance a single workload needs to be able to solely run on a host, something like SAP for instance. In order to achieve this, you can use anti-affinity rules. We are not talking about regular anti-affinity rules. This is the very first time a brand new mechanism is used on-premises. I am talking about compute policies. Compute policies have been available for VMware Cloud on AWS customers for a while, but now are also appear to be coming to on-prem customers. What does it do? It enables you to create “anti-affinity” rules for vCLS VMs and specific other VMs in your environment by creating Compute Policies and using Tags!

How does this work? Well, you go to “Policies and Profiles” and then click “Compute Policies”. Now you can click “ADD” and create a policy. You now select “Anti Affinity with vSphere Cluster Services (vCLS) VMs”. Then you select the Tag you created for the VMs that should not run on the same hosts as the vCLS VMs, and then you click create. The vCLS VM Scheduler will then ensure that the vCLS VMs will not run on the same hosts as the tagged VMs. If there’s a conflict, the vCLS Scheduler will move away the vCLS VMs to other hosts within the cluster. Let’s reiterate that, the vCLS VMs will be vMotioned to another host in your cluster, the tagged VMs will not be moved!

Hope that helps!

Issue adding tags to the vCLS VMs with vCenter Server 7.0 U2b

· ·

Today I was talking to one of our field folks and he asked if there was an issue with Tags in combination with vCLS VMs in 7.0 U2b specifically. I had tested assigning tags to vCLS VMs before, and it worked just fine. With 7.0 U2b unfortunately this has stopped working. The error you will see displayed in the vSphere Client is the following:

(vmodl.fault.SecurityError) {
faultCause = null,
faultMessage = null
}

Or as it shows in the UI:

Issue adding tags to the vCLS VMs with vCenter Server 7.0 U2b

So what can you do about it? Well, unfortunately not much right now, I filed a bug and uploaded the logs, engineers are looking at it as we speak, and hopefully, I will have an answer for those who need to use tags soon.

UPDATE: Engineering has found a workaround, customers who can’t wait for the fix can contact GSS to get the workaround implemented!

vCLS VMs not powering on, insufficient resources error

· ·

This week I had someone internally me asking about a situation where vCLS VMs (learn more about vSphere Cluster Service here.) were not powering on and an error was thrown stating “insufficient resources”. I had seen this issue before at some point and I knew it had something to do with the VM version and EVC. The details of the error messages seem to support that. The UI showed the following on the “power on virtual machine” task:

Insufficient resources

And then when you would look at the details of the error you could see the following:

The target host does not support the virtual machine's current hardware requirements.

Or you could see:

Feature 'MWAIT' was absent, but must be present.

vCLS VMs not powering on, insufficient resources error

So how do you solve this problem? First of all, this could be two different problems. We solved it the following way, please note that the second option was just us fiddling around to get the VMs provisioned and powered-on, and this is not the official VMware procedure to get it working. I have reported this to the engineers to figure out why this happens, and to get it fixed. There are two options, please use Option 1, as this is a requirement for EVC and the recommended method when you see the “MWAIT” error:

Option 1:

Verify if “Monitor/MWAIT” is set to Enabled in the BIOS. If it is set to Disabled, then this is why the power-on fails. vCLS has per-VM EVC enabled on the VM.

If you can’t enable Monitor/MWAIT, then below is the procedure for disabling “per VM EVC” for the provisioned vCLS VMs.

Option 2:

  1. Upgrade the VM’s “Compatibility” version to at least “VM version 14” (right-click the VM)
  2. Click on the VM, click on the Configure tab and click on “VMware EVC”
  3. Click on “Edit” and click on “Yes” when you are informed to not make changes to the VM
  4. Disable “EVC”
  5. Repeat for the other vCLS VMs

I want to mention cosmin.gq, as it seems the issue (and resolution with regards to disabling EVC) was also reported on that blog, and considering they reported it in October already it only seems fair to mention them here also.

How to login to the vCLS VMs!?

· ·

I was asked this question this week, how you can login to the vCLS VMs. Now before I share the video, I want to mention that I do not encourage people doing this, but as it is documented and supported I do want to provide a simple “how to” for how this works. If you want to login to the vCLS VM, maybe for troubleshooting if needed or for auditing, you can do so by SSH’ing first into your vCenter Server. When logged in to the vCenter Server you run the following command, which then returns the password, this will then allow you to login to the console of the vCLS VM. Again, I do not want to encourage you to do this. Either way, below you find the command for retrieving the password, and a short demo of me retrieving the password and logging in.

/usr/lib/vmware-wcp/decrypt_clustervm_pw.py

 

Did you know vSphere 7.0 Update 1 also has a Skyline Health Check for vSphere Clustering Services?

· ·

I did not know this, but yesterday the PM for vCLS reached out to me and informed me that we now have a Skyline Health Check as well for vSphere Clustering Services. The funny thing is that I actually requested this health check to be added after having a discussion on the topic of vCLS with the PM. Very impressive how fast the engineering team managed to include an additional health check for a brand new feature, this close to the release. I created a short demo, which shows you where you can find the vSphere Skyline Health option in the vSphere Client, and of course, it shows the vCLS Health Check being triggered. If you see the health check triggered, you can as mentioned enable retread mode and disable it again, this will provision a fresh set of vCLS VMs. How you do this you can find in this “considerations blog“, or simply watch the demo I shared here.