I had a question about a limit a customer was hitting with the VM notes using the vSphere (H5) Client. They noticed that whenever they got around ~260 or so characters, they could not add any characters beyond that unless they deleted some. After talking to an engineer within VMware I found out that this is indeed the limit for the vSphere Client today. Through the API, and of course also PowerCLI, you can go beyond the 255 if needed. Also with the Web Client you could do this. If you are hitting this issue, please be aware that it is a known issue and VMware is looking to resolve it hopefully soon.
Management & Automation
Probably one of the most downloaded tools ever, just released a new version: RVTools 4.0.4. Rob emailed me over the weekend, that this new version was just released and he also mentioned that he has reached over 1.2 million downloads. 1.2 million indeed! Crazy number, especially knowing how many customers VMware has today. Congratulations Rob! Anyway, RVTools 4.0.4 was just released, what is in it?
Version 4.0.4 (May 1, 2020)
- Upgraded RVTools solution to use VMware vSphere Management SDK 7.0
- Upgraded RVTools solution to use CIS REST API, available since vSphere 6.5, to get tag information
- vInfo tab page new columns: Virtual machine tags and min Required EVC Mode Key
- vCPU tab page new columns: Virtual machine tags
- vMemory tab page new columns: Virtual machine tags and Memory Reservation Locked To Max
- vDisk tab page new columns: Virtual machine tags
- vPartition tab page new columns: Virtual machine tags
- vCD tab page new columns: Virtual machine tags
- vFloppy tab page new columns: Virtual machine tags
- vNetwork tab page new columns: Virtual machine tags
- vSnapshot tab page new columns: Virtual machine tags
- vTools tab page new columns: Virtual machine tags
- vRP tab page new columns: Resource Pool tags and object ID
- vCluster tab page new columns: Cluster tags, custom attributes and object ID
- vHost tab page new columns: Host tags, in Maintenance Mode and in Quarantine Mode
- dvSwitch tab page new columns: Distributed VirtualSwitch tags, custom attributes and object ID
- dvPort tab page new columns: Distributed VirtualSwitch Port Group tags and object ID
- vDatastore tab page new columns: Datastore tags, custom attributes and object ID
- Preference extra checkbox for “Exclude tags”
- CLI new parameter -ExcludeTags
- Bug fix: removed column “Config Checksum” from vInfo tab page.
This Base64Binary field was sometimes the cause for a XML deserialize error!
- Bug fix: not all snapshots from all snapshot siblings where displayed
- Bug fix: preference setting “Exclude Annotation fields” value was overwritten.
- Bug fix: Name on vRP tab page was full path
- Bug fix: In example RVToolsBatchMultipleVCs.ps1 script parameters changed for RVToolsMergeExcelFiles.exe
- Bug fix: vSphere 7, “Unhandled exception: Input string was not in a correct format.”
- Known issue: Tags are only visible when logged on with userid/password. It’s not working when SSO is used!
Right after we finished recording the Virtually Speaking Podcast on the topic of VMware Tools (Listen to it, great episode featuring Pete, John, William Lam and myself) yesterday I received an email from Rob. Rob mentioned an update to RVTools, bringing it now to version 3.11.6. As I mentioned on the podcast, RVTools has been around for 10 years now, what an achievement! Insane number of downloads, but understandable as it is very useful for anyone and everyone running a VMware environment. If you never looked at it, download it today, I am sure you will find various inconsistencies or issues, we all have! So, what changed in 3.11.6?
Version 3.11.6 (March, 2019)
- Upgraded RVTools solution to use VMware vSphere Management SDK 6.7U1
- Windows Authentication Framework (Waffle) is no longer used by RVTools
- NPOI .NET library for creating excel export files is no longer used by RVTools
- RVTools now uses OpenXML and ClosedXML for creating the excel export files
- Performance improvements for export to excel
- added -ExcludeCustomAnnotations switch to RVTools command line interface
- added –DBColumnNames switch to RVTools command line interface
- vInfo tab page new column: Creation date virtual machine
- vInfo tab page new columns: Primary IP Address and vmx Config Checksum
- vInfo tab page new columns: log directory, snapshot directory and suspend directory
- dvSwitch tab page new columns: LACP name, LACP mode and LACP loadbalance Algorithm
- vNIC tab page new column: Name of uplink port
- vNetwork tab page new column: Network Adapter DirectPath I/O Parameter
- vHost tab page new columns: Serial number and BIOS vendor
- Header row and first column in export Excel file are now locked.
- First “Select” column is removed from excel worksheet vFloppy, vCD and vTools.
- added a new executable to merge your vCenter xlsx files super-fast to one xlsx file.
RVToolsMergeExcelFiles.exe -input c:\temp\AA.xlsx;c:\temp\BB.xlsx -output c:\temp\AABB.xlsx -template c:\temp\mytemplate.xlsx -verbose –overwrite
- Example script RVToolsBatchMultipleVCs.ps1 is changed. It will now uses RVToolsMergeExcelFiles to merge the xlsx files.
- Bug Fix: a Single Sign On problem solved
- Bug Fix: ExportvSC+VMK2csv command was not working
- Bug Fix: ExportdvPort2csv command was not working
- Bug Fix: On vNIC tabpage not all Switch/dvSwitch information was displayed
- Bug Fix: Export now reflect value of “Latency Sensitivity” enumeration
- Bug Fix: After changing the preference settings the data is not always refreshed as needed
- Bug fix: Content Libraries vmdk files are no longer reported as possible zombie files
Last week I was briefed by Runecast (together with Cormac) on the new version, Runecast 2.0, which was released/announced today. I always enjoy talking to Stan as every time we talk they have something new which surprises me, or he tells me about something cool on the roadmap. For those who did not read my previous articles, Runecast is a company which focusses on analyzing VMware environments and assess the environment on potential issues. These issues could be anything ranging from configuration issues, driver/firmware issue, to security issues. It reminds me very much of what we have with vSAN which is the health check. The big difference though is that this solution includes many more checks and doesn’t just focus on vSAN but on many different parts of the stack. Just to give you an idea, today Runecast can analyze your vSphere environment up to vSphere 6.7 and can also analyze vSAN and NSX-V. The cool thing is that it also does this “offline”, they have an appliance and regular updates (rules and features) and this means that even in a dark site this would work.
A lot of Runecast’s customers are either in the financial space or government space. I guess this is also why their focus for the 2.0 version was primarily on PCI-DSS. With over 200 technical checks, which map against PCI-DSS requirements, they (as Runecast told me) have by far the largest collection of requirements in an automated analyzer (for VMware) in the industry. Definitely, a smart enhancement, if you are not interested in PCI-DSS, you can simply disable the whole check and it will never show up in your interface. You can also, if only a limited number of clusters should be validated, filter out certain results.
The 20 version of Runecast also comes with a lot of updates around the appliance, now I consider these “internals” as for most customers it is not relevant in terms of the value it offers, but it is important to know from a security perspective I guess.
This version also introduces a historical perspective. Meaning that starting with Runecast 2.0 the historical information of checks is stored. This will allow you to see some form of trending when it comes to the different checks/validations. You could for instance now track if you do updates and maintenance if the number of potential issues is going down. You could also task someone with validating the reported issues and fixing those when or where possible. This should over time improve the availability, reliability, and security of your environment.
Last but not least the UI has been fully overhauled. They redesigned it just to make it easier to read and understand. Also, a couple of dashboards were added, which makes sense… a new release means new dashboards!
If you happen to go to VMworld, make sure to stop by their booth and have a look, I think you will find it interesting. Or simply read the Runecast blog, and download the appliance and try it out.
At a VMUG a couple of months ago I bumped into my old friend Dennis Zimmer. Dennis told me that he was working on something cool for vSAN but couldn’t reveal what it was just yet. Last week I had a call with Dennis about what that thing was. Dennis is the CEO for Opvizor, and some of you may recall the different tooling that Opvizor has produced over the years, of which the Health Analyzer was probably the most famous one back then. I’ve used it in the past on various occasions and I had various customers using it. During the briefing, Dennis explained to me that Opvizor started focussing on performance monitoring and analytics a while ago as the health analyzer market was overly crowded and had the issue that is was a one-off business (checks once in a while instead of daily use). On top of that, many products now come with some form of health analysis included. (See vSAN for instance.) I have to agree with Dennis, so this pivot towards Performance Monitoring makes much sense to me.
Dennis explained to me how they are seeing more and more customer demand for vSAN performance monitoring especially combined with VMware ESXi, VM and App data. Although vCenter has various metrics, and there’s VROps, he told me that Opvizor has many customers who need more than vCenter or vROPS standard has to offer today and don’t own VROps advanced. This is where Opvizor Performance Analyzer comes in to play and that is why today Opvizor announced they are including vSAN specific dashboards. Now, this isn’t just for vSAN of course. Opvizor Performance Analyzer includes not just vSAN but also vSphere and various other parts of the stack. When talking with Dennis one thing became clear, Opvizor is taking a different approach than most other solutions. Where most focus on simplifying, hiding, and aggregating, the focus for Opvizor is on providing as much relevant detail as possible to fulfill the needs of beginner and professional.
So how does it work? Opvizor provides a virtual appliance. You simply deploy it in your environment and connect it to vCenter and you are ready to go. The appliance collects data every 5 minutes (but 20 seconds intervals of these 5 minutes) and has a retention of up to 5 years. As I said, the focus is on infrastructure statistics and performance analytics and as such Opvizor delivers all the data you ever need.
It doesn’t just provide you with all the info you will ever need. It will also allow you to overlay different metrics, which makes performance troubleshooting a lot easier, and will allow you to correlate and pinpoint particular problems. Opvizor comes with dashboards for various aspects, here are the ones included in the upcoming release for vSAN:
- Capacity and Balance
- Storage Diskgroup Stats
- VM View
- Physical disk latency breakdown
- Cache Diskgroup stats
- vSAN Monitor
Now I said this is the expert´s troubleshooting tool, but Opvizor Performance Analyzer also provided in-depth information about what each metric is / means and provides starter dashboards for beginners. You can simply click on the “i” in the top left corner of the widget and you get all the info about that particular widget.
When you do know what you are looking for you can click, hover, and zoom when needed. Hover over the specific section in the graph and the point in time values of the metrics will pop up. In the case below I was drilling down on a VM in the vSAN cluster and looking at write latency in specific. As you can see we have 3 objects and in particular 2 disks and a “vm name space”.
And this is just a random example, there are many metrics to look at and many different widgets and overviews. Just to give you an idea, here are some of the metrics you can find in the UI:
- Latency (for all different components of the stack)
- IOPs (for all different components of the stack)
- Bandwidth (for all different components of the stack)
- Congestion (for all different components of the stack)
- Outstanding I/O (for all different components of the stack)
- Read Cache Hit rate (for all different components of the stack)\
- ESXi vSAN host disk usage
- ESXi vSAN host cpu usage
- Number of Components
- Disk Usage
- Cache Usage
And there;s much more, too many to list in this blog. And again, not just vSAN, but there are many dashboards to chose from. If you don’t have a performance monitoring solution yet and you are evaluating solutions like SolarWinds, Turbunomics and others make sure to add Opvizor to that list. One thing I have to say, I spotted a couple of things that I liked to see changed, and I think within 24hrs the Opvizor guys managed to incorporate the feedback. That was a crazy fast turnaround, good to see how receptive they are.
Oh, one more thing I found in the interface, it is these dashboards that deal with things like NUMA. But also things like the Top 10 VMs in terms of IOPS. Both very useful, especially when doing deep performance troubleshooting and optimizing.
I hope that gives you a sense of what they can do. There’s a fully functional 30-day trial, check it out if you want to find out more about Performance Analyzer or simply just want to play around with it. Opvizor announced this brand new version on their own blog here, make sure to give that a read as well!