Virtual Events I will be presenting/speaking at the upcoming months

Duncan Epping · Nov 19, 2020 · 2 Comments

The past year has been really strange for me. As you know a lot of time goes into speaking at events, or doing a series of customer meetings. Due to COVID that all rapidly changed in March of this year. Fortunately, it didn’t mean I was out of work, we had to figure some stuff out in the first few weeks, but quickly everyone shifted towards this virtual approach. We (Cormac, Frank, and I) have presented at some great events the past 8 months, and there are some very interesting events coming which I will be speaking at (some with Cormac and Frank, and some without them).

Even though many of these events are listed/advertized as regional events, as they are all virtual it means anyone can tune in. I wanted to share a few events I have planned before Christmas, which could be worth attending! In some cases my session is in Dutch, but I called that out in the list.

1-December >> Define Tomorrow Keynote – https://live.computerworld.co.uk/talks/keynote-how-hci-is-revolutionizing-the-datacenter-today-and-tomorrow/
3-December >> VMware vSAN 7.0 U1 Webinar – https://www.vmware.com/learn/695402_EN_REG.html?src=so_5fa4046b6e733
8-December >> VMUG Usercon Nederland Breakout – https://vmugvirtualnlusercon.vfairs.com/ (Note, my session will be in Dutch)
10-December >> VMUG Usercon Portland Keynote – https://vmugvirtualportlandusercon.vfairs.com/

So if you are interesting in hearing about things like vSAN, Cloud Native Storage, vSphere, and much more, make sure to sign up for one of these events!

How to login to the vCLS VMs!?

Duncan Epping · Nov 17, 2020 · 2 Comments

I was asked this question this week, how you can login to the vCLS VMs. Now before I share the video, I want to mention that I do not encourage people doing this, but as it is documented and supported I do want to provide a simple “how to” for how this works. If you want to login to the vCLS VM, maybe for troubleshooting if needed or for auditing, you can do so by SSH’ing first into your vCenter Server. When logged in to the vCenter Server you run the following command, which then returns the password, this will then allow you to login to the console of the vCLS VM. Again, I do not want to encourage you to do this. Either way, below you find the command for retrieving the password, and a short demo of me retrieving the password and logging in.

/usr/lib/vmware-wcp/decrypt_clustervm_pw.py

Which vSAN policy changes will trigger a rebuild?

Duncan Epping · Nov 10, 2020 · 2 Comments

A couple of years ago I did a VMworld session with Cormac and we discussed the top things everyone should know about vSAN. One of the items discussed was which policy changes would trigger a rebuild. We tested the various situations and documented them. Two weeks ago a question around this was asked on a VMware internal Slack channel so I shared our findings. Considering it is already a few years ago, I wanted to make sure that our documented findings were still valid, so I redid the tests.

Now before I provide a table with the findings, I just want to explain what I tested, what I did is I created a VM with a default policy. I dumped a bunch of random data on the two VMDKs attached to the VM, and I then changed the policy of the VM while the VM is running. After changing the policy I verified through the command-line, and UI, if a rebuild of the objects was occurring or not. In some cases a policy change does not require a rebuild, while in other cases it does. This, of course, depends on what is being changed within the policy, and what that means for the objects associated with the policy. Hopefully, you will find the below table useful.

From	To	Resync
RAID-1	RAID-1 with higher FTT	Yes
RAID-1	RAID-1 with lower FTT	No
RAID-1	RAID-5/6	Yes
RAID-5/6	RAID-1	Yes
RAID-5	RAID-6	Yes
RAID-6	RAID-5	Yes
Stripe width 1	Stripe width increase by 1 (or more)	Yes
Stripe width x	Stripe width decrease by 1 (or more)	Yes
Space Reservation 0	Increase to larger than 0	No
Space Reservation >= 1	Increase by 1 (or more)	No
Space reservation > 0	Decrease to 0	No
Read Cache 0	Increase to larger than 0	No
Read Cache >= 1	Increase by 1 (or more)	No
Read Cache >= 1	Decrease by 1 (or more)	No
Checksum enabled	Checksum disabled	No
Checksum disabled	Checksum enabled	Yes

Did you know vSphere 7.0 Update 1 also has a Skyline Health Check for vSphere Clustering Services?

Duncan Epping · Nov 6, 2020 · 1 Comment

I did not know this, but yesterday the PM for vCLS reached out to me and informed me that we now have a Skyline Health Check as well for vSphere Clustering Services. The funny thing is that I actually requested this health check to be added after having a discussion on the topic of vCLS with the PM. Very impressive how fast the engineering team managed to include an additional health check for a brand new feature, this close to the release. I created a short demo, which shows you where you can find the vSphere Skyline Health option in the vSphere Client, and of course, it shows the vCLS Health Check being triggered. If you see the health check triggered, you can as mentioned enable retread mode and disable it again, this will provision a fresh set of vCLS VMs. How you do this you can find in this “considerations blog“, or simply watch the demo I shared here.

vSphere HA configuration for HCI Mesh!

Duncan Epping · Oct 29, 2020 · 7 Comments

I wrote a vSAN HCI Mesh Considerations blog post a few weeks ago. Based on that post I received some questions, and one of the questions was around vSphere HA configurations. Interestingly I also had some internal discussions around how vSAN HCI Mesh and HA were integrated. Based on the discussions I did some testing just to validate my understanding of the implementation.

Now when it comes to vSphere HA and vSAN the majority of you will be following the vSAN Design Guide and understand that having HA enabled is crucial for vSAN. Also when it comes to vSAN configuring the Isolation Response is crucial, and of course setting the correct Isolation Address. However, so far there’s been an HA feature which you did not have to configure for vSAN and HA to function correctly, and that feature is VM Component Protection aka APD / PDL responses.

Now, this changes with HCI Mesh. Specifically for HCI Mesh the HA and vSAN team have worked together to detect APD (all paths down) down scenarios! When would this happen? Well if you look at the below diagram you can see that we have “Client Clusters” and a “Server Cluster”. The “Client Cluster” consumes storage from the “Server Cluster”. If for whatever reason a host in the “Client Cluster” loses access to the “Server Cluster”, it results in the VMs on that host consuming storage on the “Server Cluster” to lose access to the datastore. This is essentially an APD (all paths down) scenario.

Now, to ensure the VMs are protected by HA for this situation you only need to enable the APD response. This is very straight-forward. You simply go to the HA cluster settings and set the “Datastore with APD” setting to either “Power off and restart VMs – Conservative” or “Power off and restart VMs – Aggressive”. The difference between conservative and aggressive is that with conservative HA will only kill the VMs when it knows for sure the VMs can be restarted, wherewith aggressive it will also kill the VMs on a host impacted by an APD while it isn’t sure it can restart the VMs. Most customers will use the “Conservative Restart Policy” by the way.

As I also mentioned in the HCI Mesh Considerations blog, one thing I would like to call out is the timing for the APD scenario: The APD is declared after 60 seconds, after which the APD response (restart) is triggered automatically after 180 seconds. Mind that this is different than with an APD response with traditional storage, as with traditional storage it will take 140 seconds before the APD is declared. You can, of course, in the log file see that an APD is detected, declared and VMs are killed as a result. Note that the “fdm.log” is quite verbose, so I copied only the relevant lines from my tests.

APD detected for remote vSAN Datastore /vmfs/volumes/vsan:52eba6db0ade8dd9-c04b1d8866d14ce5
Go to terminate state for VM /vmfs/volumes/vsan:52eba6db0ade8dd9-c04b1d8866d14ce5/a57d9a5f-a222-786a-19c8-0c42a162f9d0/YellowBricks.vmx due to APD timeout (CheckCapacity:false)
Failover operation in progress on 1 Vms: 1 VMs being restarted, 0 VMs waiting for a retry, 0 VMs waiting for resources, 0 inaccessible vSAN VMs.

Now for those wondering if it actually works, of course, I tested it a few times and recorded a demo, which can be watched on youtube (easier to follow in full screen), or click play below. (Make sure to subscribe to the channel for the latest videos!)

I hope this helps!