vSphere Flash Read Cache and esxcli

As most features these days in vSphere you can configure them using the awesome esxcli command. I’ve already mentioned esxcli in my vSphere Flash Read Cache FAQ blog but I wanted to call it out explicitly here as I found it very useful. You can get some nice details using the esxcli command. So where do we start?

First thing would be:

esxcli storage vflash

This will return that there are 3 namespaces: cache, module and device. Lets start top down with device. The command “esxcli storage vflash device list” will show you a list of all flash devices and whether it has been configured for vFRC or not. The module namespace can provide you some more details around for instance cache blocksizes etc. If you run the command this is what the output looks like:

~ # esxcli storage vflash module get
 Min Supported Module Version: 1.0.0.0
 Revision: 1.0.0.0
 Supported Cache Block Size Max: 1048576
 Supported Cache Block Size Min: 4096
 Supported Cache Size Max: 214748364800
 Supported Cache Size Min: 4194304
 Supported Disk Size Max: 17592186044416
 Supported Mode Mask: WriteThru

[Read more...]

Pinging from different VMkernel NICs using esxcli?

Today I had a network issue in my lab, I still don’t have a clue what the issue was but I did discover something useful. I had 3 different VMkernel’s setup and I wanted to make sure each of the three had network connection to a specific destination address. After going through the esxcli command I bumped in to the following command which I found very helpful:

esxcli network diag ping -I vmk0 -H 10.27.51.132

In this case I use VMkernel Interface “vmk0″ to ping to the address “10.27.51.132″. If I want to use a different VMkernel Interface I just specify it, so swap “vmk0″ with “vmk1″ for instance. Useful right?!

How to change the IP Address of ESXi through the commandline

I was building out my virtualized lab and instead of re-installing ESXi over and over again I figured I would just quickly clone them. Now of course this leads to a “minor” problem as the virtualized ESXi hosts will all boot with the same IP-Address. As I don’t have DHCP to my disposal I needed to change them manually, so how do you change the IP address of ESXi through the commandline?

It is actually pretty straight forward with esxcli these days. First thing I did was listing all VMkernel NICs:

esxcli network ip interface ipv4 get

This will give you the list of all VMkernel interfaces with their details (See screenshot below). Changing the IP address is just a matter of adding some parameters:

esxcli network ip interface ipv4 set -i vmk1 -I 10.27.51.143 -N 255.255.255.0 -t static

In your situation you will need to replace “vmk1″ with the appropriate VMkernel NIC of course and change the IP details.

change ip address of esxi

How to disable ESXi firewall

For a project I had to disable the ESXi firewall on a host permanently. To be honest, it isn’t something I would do normally or would recommend even. It wasn’t listed in “chkconfig”, which kinda makes sense, so I looked at the networking section of esxcli. What an awesome command by the way! Quickly after “tab’ing” through esxcli I figured out how to disable it permanently:

esxcli network firewall set --enabled false

I figured I would write it down, because this is the stuff I tend to forget easily.

PS: If you ever need anything around esxcli, the vSphere Blog is a good place to check as most of the relevant posts are tagged with “esxcli”.

Faking an SSD in your virtualized vSphere lab

I have written about this before (and so has William Lam, so all credits go to William), but I wanted to note down these commands for my own use as I find myself digging around often for the same commands these days. So what is my goal: Faking an SSD in my virtualized vSphere lab.

In my lab I have a bunch of virtualized ESXi hosts. Those hosts have multiple disks and I want to mark one of those disks as SSD. To keep things simple I set things up as follows. Just to point out, I use 0:0 / 1:0 / 2:0 so that each device gets a new controller and is easy to identifiy:

  • First Disk – ESXi install disk – 5GB – SCSI 0:0
  • Second Disk – Fake SSD – 40GB – SCSI 1:0
  • Third Disk – Large disk – 1TB – SCSI 2:0

When I boot all disks are recognized as regular disks and in some cases as non-local. In my testing I need local disks and need SSD. So this is what I did to get exactly that. With the first command I mark the “second disk” as SSD and local. With the second command I mark the third disk as local. Next I reclaim the devices so that the new SATP rules are applied.

esxcli storage nmp satp rule add --satp VMW_SATP_LOCAL --device mpx.vmhba2:C0:T0:L0 --option "enable_local enable_ssd"
esxcli storage nmp satp rule add --satp VMW_SATP_LOCAL --device mpx.vmhba3:C0:T0:L0 --option "enable_local"
esxcli storage core claiming reclaim -d mpx.vmhba2:C0:T0:L0
esxcli storage core claiming reclaim -d mpx.vmhba3:C0:T0:L0

Next you can simply validate if it has worked by typing the following for device vmhba2 and 3 (if you replace the 2 with a 3 ofcourse) :

esxcli storage core device list --device=mpx.vmhba2:C0:T0:L0

As you can see, faking an SSD is fairly straight forward. Note that even if you have an SSD drive you still might need to do this. In some cases the SSD drive is not recognized and you will need to create a rule for it manually.

Some more nuggets about handling VIB files

After I posted my article yesterday Jason Boche posted a comment about the reboot that was required according to the screenshot. I looked in to it and quickly realized that if I would alter my “descriptor.xml” I would not get this message. In other words, it depends on the package that is installed if a reboot is required or not, in my case I made the following simple changes to install the package without the need to reboot:

<live-install-allowed>true</live-install-allowed>
<live-remove-allowed>true</live-remove-allowed>

In other words, I am allowed to install it without a reboot and remove it without a reboot. Simple huh? Of course I tested it and this is the result:

~ # esxcli software vib install -v /test.vib
Installation Result
   Message: Operation finished successfully.
   Reboot Required: false
   VIBs Installed: Duncan_bootbank_firewallrule_1.0
   VIBs Removed:
   VIBs Skipped:
~ #

After clicking refresh in the vCenter client the firewall rule I created pops up as expected.

Now if you would like to know before installing what the package contains and what the requirements are you can simply figure that out by doing the following:

~ # esxcli software sources vib get -v file:/test.vib
Duncan_bootbank_firewallrule_1.0
   Name: firewallrule
   Version: 1.0
   Type: bootbank
   Vendor: Duncan
   Acceptance Level: CommunitySupported
   Summary: Firewall rule
   Description: Firewall rule
   Release Date: 2011-06-01
   Depends:
   Conflicts:
   Replaces:
   Provides:
   Maintenance Mode Required: False
   Hardware Platforms Required:
   Live Install Allowed: True
   Live Remove Allowed: True
   Stateless Ready: False
   Overlay: False
   Tags: driver, module
   Payloads: test
~ #

As you can see in this case, “live install allowed” is set to true. The vendor is “Duncan” and the Acceptance Level is “CommunitySupported”, these are important details in my opinion! Another one to keep an eye on is if the package is “Stateless Ready” or not. In my case I defined it as “false”.

Of course you can also remove a VIB file after installing it. This is pretty straight forward, first of all list all the installed VIBs:

~ # esxcli software vib list
Name                  Version                             Vendor  Acceptance Level  Install Date
--------------------  ----------------------------------  ------  ----------------  ------------
ata-pata-amd          0.3.10-3vmw.500.0.0.456551          VMware  VMwareCertified   2011-06-06
ata-pata-atiixp       0.4.6-3vmw.500.0.0.456551           VMware  VMwareCertified   2011-06-06
ata-pata-cmd64x       0.2.5-3vmw.500.0.0.456551           VMware  VMwareCertified   2011-06-06

After listing all installed VIBs you can easily remove them by using the following command:

~ # esxcli software vib remove -n ata-pata-amd

This would remove the VIB named “ata-pata-amd”. You could even do a “dry-run” to see what the result would be:

~ # esxcli software vib remove -n ata-pata-amd --dry-run
Removal Result
   Message: Dryrun only, host not changed. The following installers 
   will be applied: [BootBankInstaller]
   Reboot Required: true
   VIBs Installed:
   VIBs Removed: VMware_bootbank_ata-pata-amd_0.3.10-3vmw.500.0.0.456551
   VIBs Skipped:
~ #

I hope this provides some more details around how to handle VIB files. Don’t hesitate to leave a comment if you have any questions at all.

How to create your own .vib files

** Be warned, this is totally unsupported. Only for educational purposes should this be used **

Today I was asked the question on how to create a VIB file (.vib). In our documentation it is mentioned that you can create a VIB file to add firewall rules to your ESXi host. As the .vib tool is not available yet to the general public I decided to dig in to it. I want to stress that I tested this in my own lab, it is not supported at all, but might give a nice insight in how these VIB are constructed. Before you read how I created my own VIB file I suggest reading this excellent article on what a .vib file is and contains by my colleague Kyle Gleed.

First thing I did was download an existing VIB file. I downloaded a tiny LSI SCSI driver. I did a “more” of the .vib file and I noticed the following:

!<arch>
debian-binary

That was my first lead, it appears to be a debian-binary, which is a format that the Linux distribution Debian uses to package software / drivers etc. I knew it should be possible to check what was included in this package. So I did a quick search and stumbled on some procedures on how to do this using some standard commands provided by my Debian virtual machine. (Links at the bottom) So I did the following on the package I downloaded:

ar tv file.vib

This showed me that the .vib file contained three files:

descriptor.xml
sig.pkcs7
scsi-meg

This seemed pretty obvious to me after reading Kyle’s article. The descriptor contained the metadata, the “sig*” file contained the signature and the “scsi-meg” was the actual driver. I decided to extract the VIB file to look at the content of these files:

ar vx file.vib

As the permissions on the files didn’t allow me to look at them I changed the permissions on those by using “chmod”. Now what? Well let’s look at the “scsci-meg” file first. What is it? I looked at what was in the file by using the following command:

tar -tzvf scsi-meg

It contained a list of files and that is it. I decided to extract it using “tar -xzvf” and as expected it was the folder structure and files part of this driver. I figured that it wouldn’t be too difficult to create a simple package. Why not try it… Here we go. First I deleted everything in the “sig.pkcs7″ file. As Kyle mentioned in his article that community support packages can have an empty signature. I also deleted all the files and folders that were extracted from the “scsi-meg” package that I did not need. I then created a folder underneath the “/etc/vmware” structure as I wanted to create a firewall rule. (Added the folder “firewall”.)

I copied a firewall rule from my existing ESXi host and which is created by HA to my Debian VM and edited the file, the original file was “fdm.xml”. I edited and and renamed it to test.xml. I changed all ports to 7000 and changed the <id> of the service that would need to be added and saved the file in “etc/vmware/firewall”.

Now it was time to package it all up and see if it would work. I guessed that the steps required would simply be the reverse of what I did to extract it all.

tar -czvf etc/ test

I then opened up the descriptor.xml file and changed some of the fields around, most don’t seem to matter much except for the following:

Change the following key to:
<acceptance-level>certified</acceptance-level>
<acceptance-level>community</acceptance-level>
Add your list of files:
<file-list>
<file>path-to-file</file>
</file-list>
Change the name of your package and the size accordingly:
<payload name="test" type="vgz" size="809">

I wasn’t sure if that would work, but I would find out eventually I guess (yes I also tried “communitysupport” as the acceptance-level but that doesn’t work!). I also removed the checksum details from the descriptor file just in case it would be used. This is what my full descriptor file looked like:

<vib version="5.0">
<type>bootbank</type>
<name>firewallrule</name>
<version>1.0</version>
<vendor>Duncan</vendor>
<summary>Firewall rule</summary>
<description>Firewall rule</description>
<release-date>2011-06-01T22:16:31.062257+00:00</release-date>
<urls/>

<relationships>
<depends>
</depends>
<conflicts/>
<replaces/>
<provides/>
<compatibleWith/>
</relationships>

<software-tags>
<tag>driver</tag>
<tag>module</tag>
</software-tags>

<system-requires>
<maintenance-mode>true</maintenance-mode>
</system-requires>

<file-list>
<file>etc/vmware/firewall/test.xml</file>
</file-list>

<acceptance-level>community</acceptance-level>
<live-install-allowed>false</live-install-allowed>
<live-remove-allowed>false</live-remove-allowed>
<cimom-restart>false</cimom-restart>
<stateless-ready>false</stateless-ready>
<overlay>false</overlay>

<payloads>
<payload name="test" type="vgz" size="809">
</payload>
</payloads>
</vib>

Next up would be making a single .vib file out of these three components again:

ar -r test.vib test descriptor.xml sig.pkcs7

Now I need to ‘scp’ the file to my ESXi host and see if I can install it:

scp test.vib root@esxi:test.vib
esxcli software vib install -v /test.vib

I received an error that the ImageProfile acceptance level needed to be changed. That was my next step:

esxcli software acceptance set --level CommunitySupported

After repeating the “esxcli software vib install” command I received the following output:

~ # esxcli software vib install -v /test.vib
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: Duncan_bootbank_firewallrule_1.0
   VIBs Removed:
   VIBs Skipped:
~ #

I rebooted the host and here’s a screenshot of the ESXi firewall with the newly added custom service “Test”:

Once again, I want to point out that this is currently unsupported. Don’t use this in your production environment!

The following articles helped me figuring this out and producing this article:

http://tldp.org/HOWTO/html_single/Debian-Binary-Package-Building-HOWTO/

http://linuxtrove.com/wp/?p=78