Yellow Bricks

by Duncan Epping

Various

Introducing vSAN File Services as part of vSAN 7.0

· · 4 Comments

There was a great article by Cormac talking about vSAN File Services published last week. I had some articles planned, but hadn’t started yet, so when I saw the article I figured I would do something different. No point in rehashing what Cormac has already shared right. I figured I would shoot another demo, and do a brief write-up so people know what vSAN File Services is all about. In vSAN/vSphere 7.0 there’s now a new feature, and this is vSAN File Services. vSAN File Services can simply be enabled on a cluster level and provides you NFS 3 and 4.1 capabilities. The great thing about the solution is that you can create file shares and associate policies with the file shares. In other words, you can have a file share with RAID-1, RAID-5, or maybe even striping or stretched across locations when that is supported.

When you enable File Services, vSAN will deploy a number of “agent VMs” and these VMs/appliances are fully managed by vSAN. These agent VMs run Photon OS and have the file services capabilities enabled through docker/container technology. After these File Service Agent VMs have been deployed, the docker container instances have been instantiated and configured, vSAN File Services will be up and running and available for use. Next, you could simply create a file share and start consuming it. But before I reveal everything, let’s just head over to the demo below. I hope you enjoy it!

NVIDIA rendering issues? Look at the stats!

· · Leave a Comment

I’ve been down in the lab for the last week doing performance testing with virtual reality workloads and streaming these over wifi to an Oculus Quest headset. In order to render the graphics remotely, we leveraged NVIDIA GPU technology (RTX 8000 in my case here in the lab). We have been getting very impressive results, but of course at some point hit a bottleneck. We tried to figure out which part of the stack was causing the problem and started looking at the various NVIDIA stats through nvidia-smi. We figured the bottleneck would be GPU, so we looked at things like GPU utilization, FPS etc. Funny enough this wasn’t really showing a problem.

We then started looking at different angles, and there are two useful commands I would like to share. I am sure the GPU experts will be aware of these, but for those who don’t consider themselves an expert (like myself) it is good to know where to find the right stats. While whiteboarding the solution and the different part of the stacks we realized that GPU utilization wasn’t the problem, neither was the frame buffer size. But somehow we did see FPS (frames per second) drop and we did see encoder latency go up.

First I wanted to understand how many encoding sessions there were actively running. This is very easy to find out by using the following command. The screenshot below shows the output of it.

nvidia-smi encodersessions

As you can see, this shows 3 encoding sessions. One H.264 session and two H.265 sessions. Now note that we have 1 headset connected at this point, but it leads to three sessions. Why? Well, we need a VM to run the application, and the headset has two displays. Which results in three sessions. We can, however, disable the Horizon session using the encoder, that would save some resources, I tested that but the savings were minimal.

I can, of course, also look a bit closer at the encoder utilization. I used the following command for that. Note that I filter for the device I want to inspect which is the “-i <identifier>” part of the below command.

nvidia-smi dmon -i 00000000:3B:00.0

The above command provides the following output, the “enc” column is what was important to me, as that shows the utilization of the encoder. Which with the above 3 sessions was hitting 40% utilization roughly as shown below.

How did I solve the problem of the encoder bottleneck in the end? Well I didn’t, the only way around that is by having a good understanding of your workload and proper capacity planning. Do I need an NVIDIA RTX 6000 or 8000? Or is there a different card with more encoding power like the V100 that makes more sense? Figuring out the cost, performance and the trade-off here is key.

Two more weeks until the end of my Take 3 experience, and what a ride it has been. If you work for VMware and have been with the company for 5 years… Consider doing a Take 3, it is just awesome!

Can’t enable AMD ReLive VR during install of Radeon Pro Software?

· · Leave a Comment

Yesterday I bumped into an issue where I wanted to enable AMD ReLive VR but the option didn’t show in the configuration window strangely enough. I remembered that the first time I installed the Radeon Pro Software for Enterprise I had an option to enable AMD ReLive VR during the process, but I couldn’t recall seeing the option this time during the install. I simply reinstalled Radeon Pro assuming the option would pop up but it didn’t. It seems that this was caused by the fact that there were already AMD drivers installed, a bit strange as all other AMD Radeon Pro components can be selected and installed when there’s a driver present, but ReLive simply won’t show up as an option.

So I used the AMD provided tools to completely uninstall all AMD Radeon related software. When you do this and you reboot the VM you will be presented the following screen at the end of the install of the Radeon Pro software, this then allows you to install ReLive VR, which you can then configure and enable through the settings window as also shown below.

AMD Radeon settings window transparent in a VM?

· · Leave a Comment

I have been playing with VR technology for the past month. The last couple of weeks my focus has been to install/configure a VM which streams the VR app over wifi to a headset. I ran into a problem with ALVR last week as documented here, but I also ran into an issue with the AMD Radeon software when I wanted to use the AMD tools to stream a VR app. When you install the AMD Radeon software within a VM and want to configure the (passthrough) graphics card or ReLive VR the Radeon configuration window shows up transparent, it looks as below. Which means you can’t configure it, you can’t enable things like ReLive VR.

The only way to get the window to show up normal is to remove the VMware SVGA device using Device Manager. Simply completely remove it and restart the VM and the problem is solved. If you have svga.present set to false you will need to click “view hidden devices” in Device Manager first before you can remove the installed software/driver by the way. When rebooted it will look normal again and it will allow you to enable and configure ReLive VR, or any other options you need to configure of course.

Seeing green only on your HMD when using ALVR to stream an app?

· · Leave a Comment

I have been testing various things as part of the Take 3 I started not too long ago. While I was setting up my environment I ran into a few issues. One of those issues was something very strange. Just so people understand what I am testing, I have an Oculus Quest headset to which I want to stream a VR app over WIFI from a powerful VM which has a passthrough GPU. Now by default, this isn’t possible. The Quest wasn’t intended for this particular usecase. In order to do this you need to setup some kind of remoting technology, which is where ALVR comes in to play. ALVR is an open source remoting/streaming solution for VR applications. Huh, what are you doing? Well as shown in the diagram below I am basically running an App using Steam within Windows and then streaming that output using ALVR from the server to the client, where the client runs as an app on the HMD (head-mounted display).

There’s also AMD ReLive VR and NVIDIA Cloud XR by the way, of which I have also tested AMD ReLive VR, which is embedded in the AMD Driver and can be enabled through the AMD advanced settings. Anyway, while testing this solution I had to disable the display head by setting “svga.present = false” in order for ALVR to work (otherwise I would get an error stating “could not create graphics device for adapter 0”), which means that as a result, I can’t access the VM using the Web/Remote Console, unfortunately.

So in order to launch the VR app and ALVR Server I have to RDP into the Windows 10 VM. When doing so I can launch the apps and connect the Headmount to ALVR Server, great… But when putting on the headset I would only see green, basically a big green screen. So why did this happen? Well, it appears that it is an artifact caused by the fact that I am launching the VR app from within an RDP session. When using RDP you end up using a specific video driver for the screen rendering, which is not something ALVR (or AMD ReLive VR) understands. So in order to get around it, you will need to log in from a “proper” console to the Windows VM and launch the app from there so that it is rendered by the AMD or NVIDIA driver instead. I used TightVNC to get around the problem, there are other solutions, but this was the fastest to implement for me.