Yellow Bricks

by Duncan Epping

Top 10 VMware tools podcast and RVTools 3.11.6

1 Comment

Right after we finished recording the Virtually Speaking Podcast on the topic of VMware Tools (Listen to it, great episode featuring Pete, John, William Lam and myself) yesterday I received an email from Rob. Rob mentioned an update to RVTools, bringing it now to version 3.11.6. As I mentioned on the podcast, RVTools has been around for 10 years now, what an achievement! Insane number of downloads, but understandable as it is very useful for anyone and everyone running a VMware environment. If you never looked at it, download it today, I am sure you will find various inconsistencies or issues, we all have! So, what changed in 3.11.6?

Version 3.11.6 (March, 2019)

  • Upgraded RVTools solution to use VMware vSphere Management SDK 6.7U1
  • Windows Authentication Framework (Waffle) is no longer used by RVTools
  • NPOI .NET library for creating excel export files is no longer used by RVTools
  • RVTools now uses OpenXML and ClosedXML for creating the excel export files
  • Performance improvements for export to excel
  • added -ExcludeCustomAnnotations switch to RVTools command line interface
  • added –DBColumnNames switch to RVTools command line interface
  • vInfo tab page new column: Creation date virtual machine
  • vInfo tab page new columns: Primary IP Address and vmx Config Checksum
  • vInfo tab page new columns: log directory, snapshot directory and suspend directory
  • dvSwitch tab page new columns: LACP name, LACP mode and LACP loadbalance Algorithm
  • vNIC tab page new column: Name of uplink port
  • vNetwork tab page new column: Network Adapter DirectPath I/O Parameter
  • vHost tab page new columns: Serial number and BIOS vendor
  • Header row and first column in export Excel file are now locked.
  • First “Select” column is removed from excel worksheet vFloppy, vCD and vTools.
  • added a new executable to merge your vCenter xlsx files super-fast to one xlsx file.
    RVToolsMergeExcelFiles.exe -input c:\temp\AA.xlsx;c:\temp\BB.xlsx -output c:\temp\AABB.xlsx -template c:\temp\mytemplate.xlsx -verbose –overwrite
  • Example script RVToolsBatchMultipleVCs.ps1 is changed. It will now uses RVToolsMergeExcelFiles to merge the xlsx files.
  • Bug Fix: a Single Sign On problem solved
  • Bug Fix: ExportvSC+VMK2csv command was not working
  • Bug Fix: ExportdvPort2csv command was not working
  • Bug Fix: On vNIC tabpage not all Switch/dvSwitch information was displayed
  • Bug Fix: Export now reflect value of “Latency Sensitivity” enumeration
  • Bug Fix: After changing the preference settings the data is not always refreshed as needed
  • Bug fix: Content Libraries vmdk files are no longer reported as possible zombie files

DQLEN changes, what is going on?

1 Comment

I had a question this week on twitter, it was about the fact that DQLEN changes to values well below it was expected to be (30) in esxtop for a host. There was latency seen and experienced seen for VMs so the question was why is this happening and wouldn’t a lower DQLEN make things worse?

My first question: Do you have SIOC enabled? The answer was “yes”, and this is (most likely) what is causing the DQLEN changes. (What else could it be? Adaptive Queueing for instance.) When SIOC is enabled it will automatically change DQLEN when the configured latency threshold is exceeded based on the number of VMs per host and the number of shares. DQLEN will be changed to ensure a noisy neighbor VM is not claiming all I/O resources. I described how that works in this post in 2010 on Storage IO Fairness.

How do you solve this problem? Well, first of all, try to identify the source of the problems, this could be a single (or multiple) VMs, but it could also be that in general, the storage array is running at its peak constantly or backend services like replication is causing a slowdown. Typically it is a few (or one) VMs causing the load, try to find out which VMs are pushing the storage system and look for alternatives. Of course, that is easier said than done, as you may not have any expansion possibilities in the current solution. Offloading some of the I/O to a caching solution could also be an option (Infinio for instance), or replace the current solution with a more capable system is another one.

Changed advanced setting VSAN.ClomRepairDelay and upgrading to 6.7 u1? Read this…

4 Comments

If you changed the advanced setting VSAN.ClomRepairDelay to anything else than the default 60 minutes there’s a caveat during the upgrade to 6.7 U1 you need to be aware of. When you do this upgrade the default is reset, meaning the value is configured once again to 60 minutes.  It was reported on twitter by “Justin Bias” this week, and I tested in the lab and indeed experience the same behavior. I set my value to 90 and after an upgrade from 6.7 to 6.7 U1 the below was the result.

Why did this happen? Well, in vSAN 6.7 U1 we introduced a new global cluster-wide setting. On a cluster level under “Configure >> vSAN >> Services” you now have the option to set the “Object Repair Time” for the full cluster, instead of doing this on a host by host basis. Hopefully this will make your life a bit easier.

Note that when you make the change globally it appears that the Advanced Settings UI is not updated automatically. The change is however committed to the host, this is just a UI bug at the moment and will be fixed in a future release.

HA Admission Control Policy: Dedicated Failover Hosts

11 Comments

This week I received some questions on the topic of HA Admission Control. There was a customer that had a cluster configured with the dedicated failover host admission control policy and they had no clue why. This cluster had been around for a while and it was configured by a different admin, who had left the company. As they upgraded the environment they noticed that it was configured with an admission control policy they never used anywhere else, but why? Well, of course, the design was documented, but no one documented the design decision so that didn’t really help. So they came to me and asked what it exactly did and why you would use it.

Let’s start with that last question, why would you use it? Well normally you would not, you should not. Forget about it, well unless you have a specific use case and I will discuss that later. What does it do?

When you designate hosts as failover hosts, they will not participate in DRS and you will not be able to run VMs on these hosts! Not even in a two-host cluster when placing one of the two in maintenance. These hosts are literally reserved for failover situations. HA will attempt to use these hosts first to failover the VMs. If, for whatever reason, this is unsuccessful, it will attempt a failover on any of the other hosts in the cluster. For example, in a when two hosts would fail, including the hosts designated as failover hosts, HA will still try to restart the impacted VMs on the host that is left. Although this host was not a designated failover host, HA will use it to limit downtime.

As can be seen above, you select the correct admission control policy and then add the hosts. As mentioned earlier, the hosts added to this list will not be considered by DRS at all. This means that the resources go wasted unless there’s a failure. So why would you use it?

  • If you need to know where a VM runs all the time, this admission control policy dictates where the restart happens.
  • There is no resource fragmentation, as a full host (or multiple) worth of resources will be available to restart VMs on, instead of 1 host worth of resources across multiple hosts.

In some cases, the above may be very useful, for instance knowing where a VM is all the time could be required for regulatory compliance, or could be needed for licensing reasons when you run Oracle for instance.

Free E-Book: Operationalizing VMware vSAN

Leave a Comment

A while ago my colleague and friend Kevin Lees reached out to me and asked me if I could go over some material he wrote together with Paul Wiggett. He also asked me if I would be willing to write a foreword. When Kevin send the document over I literally finished it within a day. What I enjoyed most about this vSAN book was the fact that it wasn’t a deep dive, it wasn’t drilling down on technology, instead the people/process aspect of things are being discussed. This is an area which is often overlooked, and definitely an area that deserves more attention when people are looking to adopt software-defined storage, or the software-defined data center for that matter. Thanks Paul/Kevin for providing me the opportunity to write the foreword, I just downloaded my free copy and I have to say it looks great.

If you are interested, the book can be downloaded for free through the VMware Virtual Blocks blog, simply go here and download your copy.