Yellow Bricks

by Duncan Epping

vSphere

Can I boot ESXi from an SD card while placing OSDATA on my SAN?

· ·

I see this question popping up all the time, literally twice a day on our VMware internal slack, can I boot ESXi from an SD card while placing OSDATA on my SAN? I guess people are confused after reading this article. It is probably a result of skim-reading as the article is in-depth and spells it out to the letter. If you look at the following table then it mentions FCoE and iSCSI:

However, FCoE, iSCSI, and FC would only be supported when you boot from SAN, only then is OSDATA supported on a SAN device. When you boot from USB/SD, OSDATA will need to reside on a locally attached device. In other words, the answer to the original question is: no you cannot boot ESXi from an SD card and place OSDATA on your SAN. Again, for details, read this excellent document.

Project Monterey, Capitola, and NVIDIA announcements… What does this mean?

· ·

At VMworld, there were various announcements and updates around projects that VMware is working on. Three of these announcements/projects received a lot of attention at VMworld and from folks attending. These announcements were Project Monterey, Project Capitola, and the NVIDIA + VMware AI Ready Platform. For those who have not followed these projects and/or announcements, they are all about GPUs, DPUs, and Memory. The past couple of weeks I have been working on developing new content for some upcoming VMUGs, these projects most likely will be part of those presentations.

When I started looking at the projects I looked at them as three separate unrelated projects, but over time I realized that although they are separate projects they are definitely related. I suspect that over time these projects, along with other projects like Radium, will play a crucial role in many infrastructures out there. Why? Well, because I strongly believe that data is at the heart of every business transformation! Analytics, artificial intelligence, machine learning, it is what most companies over time will require and use to grow their business, either through the development of new products and services or through the acquisition of new customers in innovative ways. Of course, many companies already use these technologies extensively.

This is where things like the work with NVIDIA, Project Monterey, and Project Capitola come into play. The projects will enable you to provide a platform to your customers that will enable them to analyze data, learn from the outcome, and then develop new products, services, be more efficient, and/or expand the business in general. When I think about analyzing data or machine learning, one thing that stands out to me is that today close to 50% of all AI/ML projects never go into production. This is for a variety of reasons, but key reasons these projects fail typically are Complexity, Security/Privacy, and Time To Value.

This is why the collaboration between VMware and NVIDIA is extremely important. The ability to deploy a VMware/NVIDIA certified solution should mitigate a lot of risks, not just from a hardware point of view, but of course also from a software perspective, as this work with NVIDIA is not just about a certified hardware stack, but also the software stack that is typically required for these environments will come with it.

So what has Monterey and Capitola to do with all of this? If you look at some of Frank’s posts on the topic of AI and ML, it becomes clear that there are a couple of obvious bottlenecks in most infrastructures when it comes to AI/ML workloads. Some of these bottlenecks have been bottlenecks for storage and VMs in general as well for the longest time. What these bottlenecks are? Data movement and memory. This is where Monterey and Capitola come into play. Project Monterey is VMware’s project around SmartNICs (or DPUs as some vendors call them). SmartNICs/DPUs provide many benefits, but the key benefit is definitely the offloading capabilities. By offloading certain tasks to the SmartNIC, the CPU will be freed up for other tasks, which will benefit the workloads running. Also, we can expect the speed of these devices to also go up, which will allow ultimately for faster and more efficient data movement. Of course, there are many more benefits like security and isolation, but that is not what I want to discuss today.

Then lastly, Project Capitola, all about memory! Software-Defined Memory as it is called in this blog. This remains the bottleneck in most environments these days, with-or-without AI/ML, you can never have enough memory! Unfortunately, memory comes at a (high) cost. Project Capitola may not make your DIMMs cheaper, but it does aim to make the use of memory in your host and cluster more cost-efficient. Not only does it aim to provide memory tiering within a host, but it also aims to provide pooled memory across hosts, which will directly tie back to Project Monterey, as low latency and high bandwidth connections will be extremely important in that scenario! Is this purely aimed at AI/ML? No of course note, tiers of memory and pools of memory should be available to all kinds of apps. Does your app need all data in memory, but is there no “nanoseconds latency” requirement? That is where tiered memory comes into play! Do you need more memory for a workload than a single host can offer? That is where pooled memory comes into play! So many cool use cases come to mind.

Some of you may have already realized where these projects were leading too, some may not have had that “aha moment” yet, hopefully, the above helps you realizing that it is projects like these that will ensure your datacenter will be future proof and will enable the transformation of your business.

If you want learn more about some of the announcements and projects, just go to the VMworld website and watch the recordings they have available!

vSphere 7.0 U3 contains two great vCLS enhancements

· ·

I have written about vCLS a few times, so I am not going to explain what it is or what it does (detailed blog here). I do want to talk about what is part of vSphere 7.0 U3 specifically though as I feel these features are probably what most folks have been waiting for. Starting with vSphere 7.0 U3 it is now possible to configure the following for vCLS VMs:

  1. Preferred Datastores for vCLS VMs
  2. Anti-Affinity for vCLS VMs with specific other VMs

I created a quick demo for those who prefer to watch videos to learn these things if you don’t skip to the text below. Oh and before I forget, a bonus enhancement is that the vCLS VMs now have a unique name, this was a very common request from customers.

Why would you need the above functionality? Let’s begin with the “preferred datastore” feature, this allows you to specify where the vCLS VMs need to be provisioned to from a storage point of view. This would be useful in a scenario where you have a number of datastores that you would prefer to avoid. Examples could be datastores that are replicated or a datastore that is only intended to be used for ISOs and templates, or maybe you prefer to provision on hybrid storage versus flash storage.

So how do you fix this? Well, it is simple, you click on your cluster object. You then click on “Configure”, and on “Datastores” under “vSphere Cluster Services”. Now you will see “VCLS Allowed”, if you click on “ADD” you now will be able to select the datastores to which these vCLS VMs should be provisioned.

Next up Anti-Affinity for vCLS. You would this feature for situations where for instance a single workload needs to be able to solely run on a host, something like SAP for instance. In order to achieve this, you can use anti-affinity rules. We are not talking about regular anti-affinity rules. This is the very first time a brand new mechanism is used on-premises. I am talking about compute policies. Compute policies have been available for VMware Cloud on AWS customers for a while, but now are also appear to be coming to on-prem customers. What does it do? It enables you to create “anti-affinity” rules for vCLS VMs and specific other VMs in your environment by creating Compute Policies and using Tags!

How does this work? Well, you go to “Policies and Profiles” and then click “Compute Policies”. Now you can click “ADD” and create a policy. You now select “Anti Affinity with vSphere Cluster Services (vCLS) VMs”. Then you select the Tag you created for the VMs that should not run on the same hosts as the vCLS VMs, and then you click create. The vCLS VM Scheduler will then ensure that the vCLS VMs will not run on the same hosts as the tagged VMs. If there’s a conflict, the vCLS Scheduler will move away the vCLS VMs to other hosts within the cluster. Let’s reiterate that, the vCLS VMs will be vMotioned to another host in your cluster, the tagged VMs will not be moved!

Hope that helps!

Booting ESXi from SD/USB devices? Time to reconsider when buying new hardware!

· ·

We’ve all seen those posts from people about worn-out SD/USB devices, or maybe even experience it ourselves at some point in time. Most of you reading this probably also knew there was an issue with 7.0 U2, which resulted in USB/SD devices wearing out a lot quicker. Those issues have been resolved with the latest patch for 7.0 U2. It has, however, resulted in a longer debate around whether SD/USB devices should still be used for booting ESXi, and it seems that the jury has reached a verdict.

On the 16th of September, a KB article was published by VMware, which contains statements around the future of SD/USB devices. I can be short about it, if you are buying new hardware make sure to have a proper persistent storage device, USB/SD is not the right choice going forward! Why? The volume of reads/writes to and from the OS-DATA partition continues to increase with every release, which means that the lower grade devices will simply wear out faster. Now, I am not going to repeat word for word what is mentioned in the KB, I would just like to urge everyone to read the KB article, and make sure to plan accordingly! Personally, I am a fan of M.2 flash devices for booting. They are not too expensive(greenfield deployments), plus they can provide enterprise-grade persistent storage to store all your ESXi related data. Make sure to follow the requirements around endurance though!

My recommended sessions for VMworld 2021!

· ·

It is that time of the year again, the VMworld content catalog has been posted online! Like every year I am going to provide a list of sessions that I highly recommend watching. There are many great sessions, and I can’t list them all, so I am going to limit it to 15 sessions again. If you feel there’s a great session missing, feel free to leave a comment of course! Oh and before I forget, I suspect that at the first day of the event multiple sessions will be added (new products/features/versions etc), so keep checking the content catalog during the event as well!

I have a session with Cormac titled: “VMware vSAN – Dynamic Volumes for Traditional and Modern Applications [MCL1084]”. We will be discussing how to use vSAN for VMs and Containers. Find it here.

In no particular order:

  • Project Monterey: Present, Future and Beyond [MCL1401] by Sudhanshu Jain and Simer Singh
    SmartNICs and DPUs have been talked about a lot at last years VMworld, and slowly we are starting to see more and more uptake of it amongst customers. If you are interested in what these accelerators can do for you today and tomorrow, make sure to check this session!
  • VMware’s Vision for Storage and Data in a Multi-Cloud World [MCL2505] by Marc Fleischmann and Vijay Ramachandran
    This has traditionally always been one of my favorite sessions about the future of Storage and Data. Why? Well, in this session Marc and Vijay will give insights into what you can expect from VMware in the near (and long term) future. Who doesn’t enjoy a roadmap!
  • Make Sustainable Choices for Product Innovation, Operations: What Can I Do? [IC2794] by Joe Baguley and Adrian Cockcroft
    Two powerhouses have teamed up, Adrian Cockcroft and Joe Baguley. Not just for this session, but they have another session around sustainability (IC2792). Not only an intriguing subject, but also something which is on top of many C-level execs today.
  • Core Storage Best Practices Deep Dive [MCL2071] by Jason Massae and Cody Hosterman
    This is a VMworld top 10 session literally every year for a reason. It has great content, it is deep technical, and it is delivered by great speakers. Must see!
  • Security Deep Dive and Emerging Capabilities in VMware Cloud on AWS [SEC1362] by Ron Fuller
    I haven’t seen any sessions by Ron just yet, but the title and the description just interest me. I think that security is still overlooked, and I think that for everyone it would be useful to learn about how security offerings are implemented in public cloud environments!
  • VEBA Revolutions – Unleashing the Power of Event-Driven Automation [CODE2773] by Michael Gasch and William Lam
    It took me a while to see where VEBA would be useful, but after it clicked I have recommended various folks to check out this event-driven way of automation to solve certain problems they were experiencing. Plus, Michael Gasch and William Lam means it will be deep and excellent!
  • VDI Nerdfest 2021: Demos That Make Admins Drool [EUS1289] by Johan van Amersfoort and Jon Towles
    This session last year was packed with great demos, this alone for me is a reason to watch it again this year. I always appreciate a great demo, something that has you thinking “how did they do that?”…
  • Extreme Performance Series: Performance best practices [MCL1635] by Mark A and Valentin Bondzio
    This one comes as both a session as well as a boot camp (MCL2033). This session will provide you the tools you need to do performance troubleshooting, or simply will provide you with a better understanding of all the different parts of the stack and where potential choking points could be.
  • 60 Minutes of Non-Uniform Memory Access (NUMA) 3rd Edition [MCL1853] by Frank Denneman
    Never attended this session and you call yourself a geek? SHAME!
  • Best Practices for Running AI Workloads in VMs on VMware vSphere [VI1459] by Justin Murray and Andrew Liu
    Justin delivered one of the best sessions in 2020 if you ask me, also on the topic of AI and GPUs, and I expect a lot of this joint session with NVIDIA’s Andrew Liu.
  • The Future of VM Provisioning – Enabling VM Lifecycle Through Kubernetes [APP1564] by Myles Gray and Nikitha Suryadevara
    Kubernetes is all about containerized apps! Could it be used for provisioning VMs? Myles and Nikitha will tell you all about it in this session. Interesting concept!
  • The Evolution of Intelligent Edge and Electrical Grid Modernization [VI1455] by Daniel Beveridge, Alan Renouf, and Prithpal Khajuria
    Daniel and Alan are part of the Office of CTO, and more specifically the innovation labs. That is why this session stood out. Combine it with the buzzwords “intelligent edge” and a co-speaker from Intel (Prithpal), and I am sure this session will spark conversations!
  • Upskill Your Workforce with Augmented and Virtual Reality and VMware [VI1596] by Alan Renouf and Matt Coppinger
    I worked on a project at the start of 2020 with Alan and Matt around virtualizing VR and AR workloads, very interesting stuff. Also interesting to hear about these different AR and VR use cases that enterprises have these days. Reminds me, I still need to pick up an Oculus Quest 2.
  • Automating Ransomware Remediation with the VMware Carbon Black Cloud SDK [CODE2782] by Alex van Brunt and Emanuela Mitreva
    Open up any website that discusses IT and you will most likely see one or two articles on the topic of ransomware. There’s a long list of sessions discussing what Carbon Black is at VMworld, but this one stood out to me as it discussed automating the remediation, which should be a crucial part of your plan to defend against attacks!
  • Migration in Action with Google Cloud VMware Engine [MCL1764] by Emad Younis and Simon Long
    Emad has a few sessions on the topic of multi-cloud workload migration and management, I had to pick one, so I picked this one where the migration to Google Cloud is discussed. Sounds like an interesting session, and both Emad and Simon are great presenters, so it should be good!
  • That was it for now, if more sessions are added to the catalog I may just do an update, probably after VMworld after I watched some of them. Anyway, now it is time to sign up for VMworld 2021! Let’s hope we can all attend “in-person” again next year.