Yellow Bricks

by Duncan Epping

VMware

#STO1490BU : vSAN Vision – The Future of HCI

· ·

I am not able to attend many sessions, but wanted to add this one to my schedule. This session was hosted by Lee Caswell (VP Products Storage and Availability VMware), Christos Karamanolis (CTO Storage and Availability VMware) and the VP for Technical Solutions for Fox Dean Perrine.

Lee Caswell kicked off talking about the customer adoption and spoke about some of the fundaments of hyper-convergence and storage management. I think the numbers speak for itself that Lee spoke about, an expected marked size of 10.7 billion USD f for HCI in 2021, growing 3x faster than traditional storage systems. Key reason for it being: simplification of ops/architecture. And that is the reason we just hit 10k customers.

vSAN had 2 major releases in the past year, with some “firsts” during the past 12 months. Like first in HCI to support Intel Optane NVMe SSDs and HPE Synergy support etc. Today Lenovo released a new integrated HCI system called ThinkAgile VX. ServeTheHome has a great article on it, make sure to read it. What is the benefit over a ReadyNode? Well for instance that all support is coming from Lenovo, it is “appliance style”.

Jeff Hunter was pulled up to on stage to demo vSAN and the power of vSAN in terms of performance and ease of use. Jeff moved a VM from NFS to vSAN and shows what the performance increase was. More importantly in my opinion was how easy it was to move the VM by leveraging Policy Based Management. (Change policy and the correct datastore is presented, vSAN in this case.)

Next is announcement around “licensing”, single socket solution, 3 servers including all licenses for less than 25k. Great price point of which hopefully many SMBs and Enterprises which need a ROBO/Edge solution can benefit from.

Dean Perrine is up next and is discussing what Fox is using vSAN for and why. They replaced their NetApp FlexPod environment with Cisco UCS C220 with vSAN. Main reason for replace the FlexPod was the need to reduce complexity, lower power consumption while increasing performance. Fox had problems around Database performance, and it was critical to support this. The environment is used to run Mission-critical video applications like their broadcasting system and channels like Fox Sport etc. In other words, events like the Superbowl are hosted on vSAN as an infrastructure. Currently running across multiple sites leveraging vSAN, NSX and SRM to provide resiliency and flexibility need to run important workloads like this in a highly resilient fashion.

Up next Christos Karamanolis, so it is going to get technical. First up discussing performance, most critical for enterprise customer is predictable performance. Christos shows a benchmark that the vSAN team ran, which was a Stock Brokerage (TPC-E like) workload. What is shown is consistent latency below a millisecond across tests and database sizes. As Christos emphasizes, very important for customers like Fox, as large latency spikes could easily disrupt broadcasting large events.

Christos next demoes the Performance Diagnostic feature that is part of vSAN 6.6.1. If you haven’t seen it yet, watch my demo on youtube. The feature in short analyses benchmarks and provides you tips and hints around how to improve your benchmark or your vSAN configuration from a policy, software or hardware point of view.

Next Christos discusses vSAN usage for VMware Cloud on AWS. What was interesting here, is what Christos mentioned around scaling: independently scale vSAN / Storage from Compute. He didn’t add much detail, so I cannot share much more than this, so I will leave it up to your imagination.

Storage for Cloud Native Apps is up last. What if you have Kubernetes or are using Docker? Can you leverage vSAN or something else? Meet Project Hatchway. Which is a combination of a Docker Volume Service and the vSphere Cloud Provider for Kubernetes. Drivers for storage, natively integrated for instance with Kubo. Leveraging Policy Based Management control/manage/monitor your persistent storage for you containarized workloads. There’s a ton of detail to be found on StorageHub here. Something else that will be introduced to optimize vSAN for workloads like Cassandra / MongoDB etc is the ability to have “storage affinity” for your workloads. In other words, data is co-located with the VM and we will not move the VM around is locality is key for these workloads. You can also imagine these workloads to be deployed with a low “failures to tolerate” level as availability is provided by the app.

And that was it. Great session, thanks for that. Looking forward to sitting in on some of the other “vision/futures” vSAN related sessions.

vSAN 6.6.1 Performance Diagnostics Demo

· ·

I was playing in my lab this morning and figured I would record a demo of a new feature which is part of vSAN 6.6.1. The Performance Diagnostics feature is aimed to help those running benchmarks to optimize their benchmarks, or optimize their vSAN configuration to reach their expected goals. Note that it is a “cloud connected” feature, and in order to use this you need to participate in the Customer Experience Improvement Program. This however is enabled by default in the latest releases of vSphere. This means that data is send up to the VMware cloud, anonymous, then analyzed and the results are send back to the Web Client. Anyway, enough said, just watch the demo.

Can you use the management IPs as the isolation address for HA?

· ·

There was a question on VMTN this week about the use of the management IP’s in a “smaller” cluster as the isolation address for vSphere HA. The plan was to disable the default isolation address (default gateway) and then add every management IP as an isolation address. In this case 5 or 6 IP’s would be added. I had to think this through and went through the steps of what happens in the case of an isolation event:

  1. no traffic between secondary and primary or primary and secondary hosts (depending on whether the primary is isolated or one of the secondary hosts)
  2. if it was a secondary which is potentially isolated then the secondary will start a “primary election process”
  3. if it was the primary which is potentially isolated then the primary will try to ping the isolation addresses
  4. if it was a secondary and there’s no response to the election process then the secondary host will ping the isolation address after it has elected itself as primary host
  5. if there’s no response to any of the pings (happen in parallel) then the isolation is declared and the isolation response is triggered

Now the question is: will there be a response when the host tries to ping itself while it is isolated, as you need to add all ip-addresses to “isolation address” options for it to make sense… And that is what I tested. It will ping all isolation addresses. All but one will fail, the one that will be successful is the management IP address of the host which is isolated. (You can still ping your own IP when the NICs are disconnected even.) Leaving the VMs running as one of the isolation addresses responded.

In other words, don’t do this. The isolation address should be a reliable address outside of the ESXi host, preferably on the same network as the management.

Unbalanced Stretched vSAN Cluster

· ·

I had this question a couple of times the past month so I figured I would write a quick post. The question that was asked is: Can I have an unbalanced stretched cluster? In other words: Can I have 6 hosts in Site A and 4 hosts in site B when using vSAN Stretched Cluster functionality? Some may refer to this as asymmetric or uneven. Either way, the number of hosts in the two locations differ.

Is this supported? In short: Yes.

The longer answer: Yes you can do this, this is fully supported but you will need to keep your selected PFTT (Primary Failures To Tolerate), SFTT (Secondary Failures To Tolerate) and FTM (Failure Tolerance Method) in to account. If PFTT=1 and SFTT=2 and FTM=Erasure Coding (RAID5/6) then the minimum number of hosts per site is 6. You could however have 10 hosts in Site A while having 6 hosts in Site B.

Some may wonder why anyone would want this, well you can imagine you are running workloads which do not need to be recovered in the case of a full site failure. If that is the case you could stick these to 1 site. (You can set PFTT=0 and SFTT=1, which would result in this scenario for that particular VM.)

That is the power of Policy Based Management and vSAN, extremely flexibility and on a per VM/VMDK basis!

vSphere 6.5 U1 is out… and it comes with vSAN 6.6.1

· ·

vSphere 6.5 U1 was released last night. It has some cool new functionality in there as part of vSAN 6.6.1 (I can’t wait for vSAN 6.6.6 to ship ;-)). There are a lot of fixes of course in 6.6.1 and U1, but as stated, there’s also new functionality:

  • VMware vSphere Update Manager (VUM) integration
  • Storage Device Serviceability enhancement
  • Performance Diagnostics in vSAN

So those using vSAN 6.2 who upgraded to vSphere 6.0 U3, here’s your chance to upgrade to get all vSAN 6.5 functionality, and more!

The VUM integration is pretty cool if you ask me. First of all, when there’s a new release the Health Check will call it out from now on. And on top of that, when you go to VUM then also things like async drivers will be taken in to consideration. Where you would normally have to slipstream drivers in to the image and make that image available through VUM, we now ensure that the image used is vSAN Ready! In other words, as of vSphere 6.5 U1 Update Manager is fully aware of vSAN and integrated as well. We are working hard to bring all vSAN Ready Node vendors on-board. (With Dell, Supermicro, Fujitsu and Lenovo leading the pack.)

Then there’s this feature called “Storage Device Serviceability enhancement”. Well this is the ability to blink the LEDs on specific devices. As far as I know, in this release we added support for HP Gen 9 controllers.

And last but not least: Performance Diagnostics in vSAN. I really like this feature. Note that this is all about analyzing benchmarks. It is not (yet?) about analyzing steady state. So in this case you run your benchmark, preferably using HCIBench, and then analyze it by selecting a specific goal. Performance will be analyzed using “cloud analytics”, and at the end you will get various recommendations and/or explanations for the results you’ve witnessed. These will point back to KBs, which in certain cases will give you hints how to solve your (if there is) bottleneck.

Note that in order to use this functionality you need to join CEIP (Customer Experience Improvement Program), which means that you will upload info in to VMware. But this by itself is very valuable as it allows our developers to solve bugs / user experience issues and getting a better understanding of how you use vSAN. I spoke with Christian Dickmann on this topic yesterday as he tweeted the below, and he was really excited, he said he had various fixes going in the next vSAN release based on the current data set. So join the program!

For those how can’t wait, here’s the release notes and downloads links:

Oh and before I forget, there’s new functionality in the H5 Client for vCenter in 6.5 U1. As mentioned on this vSphere blog: “Virtual Distributed Switch (VDS) management, datastore management, and host configuration are areas that have seen a big increase in functionality”. And then some of the “max config” items had a big jump. 50k powered on VMs for instance is huge:

  • Maximum vCenter Servers per vSphere Domain: 15 (increased from 10)
  • Maximum ESXi Hosts per vSphere Domain: 5000 (increased from 4000)
  • Maximum Powered On VMs per vSphere Domain: 50,000 (increased from 30,000)
  • Maximum Registered VMs per vSphere Domain: 70,000 (increased from 50,000)

That was it for now!