Yellow Bricks

by Duncan Epping

Archives for January 2009

Storage Replication Adapter: discoverLuns…

· ·

Today I was implementing Site Recovery Manager with a colleague(Thanks Andy!!). During the configuration of the HP EVA SRA(Storage Replication Adapter) we received the following error:

discoverLuns script failed to execute properly

The error indicates that that the first part of the SRA configuration “discoverArrays” worked but when discovering the LUN’s and it’s replica’s it bailed out(23%). So after checking the config files and log files we decided to run the scriptfile, that the SRA uses, manually and see what happens.

First we created an XML file which feeds the script. The XML file contained the following, which can be copied from the SRM Log files:

<?xml version=”1.0″ encoding=”ISO-8859-1″?>
<Command>
<Name>discoverLuns</Name>
<ConnectSpec>
<Name>HP StorageWorks EVA Virtualization Adapter</Name>
<Address>san.yellow-bricks.com</Address>
<Username>user</Username>
<Password>password</Password>
</ConnectSpec>
<ArrayId>YB-SAN-01</ArrayId>
<OutputFile>C:\TEMP\SAN.Log</OutputFile>
<LogLevel>trivia</LogLevel>
</Command>

Now we were able to run the script with the XML file as input:

perl command.pl < file.xml

In our case running the script manually with the XML file as input didn’t return an error. This gave us the idea that it might be account or permissions related. During the configuration of the SRA we entered domain credentials, which were the same as the account being used during the manual run of the script. So it wasn’t the SRA account that was causing these problems.

After diving into the configuration we stumbled upon the SRM Service. The SRM service was started with a Local System account. We decided to change the account used for the Service from “Local” to a domain account… and indeed problem solved.

One would expect this to be part of the SRA documentation, but it isn’t. We contacted VMware Support and they had the same configuration running in their test environment except for the fact that they weren’t using AD authentication. In their case the Local System account just worked fine.

I’ve emailed Support all the log files and according to them our suspicion was correct. It seems to be related to the HP EVA SRA. The HP SRA seems to use the wrong account for authentication at one point during the script. Next up: Contact HP Support and let’s see if they can a) fix this or b) update their documentation.

VMDK alignment part two…

· ·

Yesterday I posted about the mbrscan and mbralign tool that NetApp created. This triggered Nick Triantos of NetApp to write an excellent detailed technical article on why you need to set the alignment:

When setup routine queries the CHS (Cylinder / Head / Sector) information of the disk from the BIOS, the CHS data doesn’t actually corresponded to anything physical. At large LUN sizes, the S number (sectors per track) is always reported as 63, so partitioning is set at 63 sector starting offset.

The problem is that 63 sectors, corresponds to 32256 bytes or 31.5KB, which does not align with storage array’s logical block size. Physical disk blocks always correspond to 512 bytes in size, but for efficiency and performance reasons, disk arrays use a logical block size unit that is a multiple of the physical block size.

For the complete story visit Nick’s Blog on the NetApp website.

Compare your hosts…

· ·

One of the most promising features in my opinion for the upcoming version of ESX definitely is “Host Profiles”. With host profiles you can ensure that each and every single ESX Host has been installed in the same way. But this feature isn’t available yet, and you would probably like to know if at least all hosts in a Cluster share the same LUN’s and/or Portgroups.

Well, it’s no surprise probably that Hugo Peeters created a script that does exactly that:

This Powershell script generates an overview of any items that are not available to every ESX server in a VMware cluster. These items might prevent your vm’s being vmotioned by DRS or restarted by HA. Pretty serious business, I’d say!

The items involved are:
1. datastores
2. LUNs (important when using Raw Device Mappings)
3. port groups

Hugo exports the output to a nice html file so no more importing to Excel needed or whatever.

Hop over to Hugo and pick up the script. The link is at the bottom of the article!

Check your VM’s alignment…

· ·

I’ve been test driving a script called “mbrscan” for a while now. Today I noticed that this script had already been released a couple of weeks ago. The cool thing about the script is that you can check the alignment of the VM’s filesystem from the outside. The script runs directly on the service console of ESX. NetApp’s Nick Triantos describes it as follows:

There’s a new utility that has been added, called mbrscan. The purpose of mbrscan is to identify whether or not a VM has properly aligned partitions. The script will run successfully under two conditions: a) The VM is powered off b) the VM has VMsnaps. The script will error out if none of the above two conditions are satisfied simply because the VMDK(s) are locked.

This can be very useful to identify possible performance issues within just a matter of seconds. Combine this command with “vmware-cmd” to create and remove snapshots and you’ll have the results for you entire environment within minutes.

Now that you just received your results the real question would be:”How am I going to modify this without destroying the data?”

At this point in time that’s not possible. But I know NetApp is working on a tool called “mbralign”. This tool will make it possible to align the filesystem from outside the VM.

The mbrscan tool is part of the “Unified Host Utilities Kit” which reached version 5. The toolkit isn’t available for everyone, you will need a now.netapp.com/ to be able to download it.

UPDATE: MBRSCAN is a binary not an editable script, and the script version has been floating around for a while now…

Orphaned vmdk’s

· ·

While doing a “mini-healthcheck” at a customer site I noticed a specific Datastore with less than 2% of free diskspace. After a bit of research an orphaned VMDK was found. Orphaned vmdk’s are virtual hard-disks that are not connected to a VM. Probably because they were removed from the inventory without deleting the files.

You can easily find these orphaned vmdk’s via the Service Console:

find -iname “*-flat.vmdk” -mtime +7

For those that don’t like using the Service Console you can also check this with Powershell Ad van Bokhoven created a nice script which he describes as follows:

This script asks the virtual center what the disk are of each VM and puts this into an array. After this, it reads all files on all datastores. If the file is a vmdk file, it will check wheter this file is in the array. If it’s not, you’ve found a orphaned vmd.

I would advise to regularly check your environment on orphaned disks, it can save precious diskspace.