Yellow Bricks

by Duncan Epping

EnableResignature and/or DisallowSnapshotLUN

· ·

I’ve spend a lot of time in the past trying to understand the settings for EnableResignature and DisallowSnapshotLUN. It had me confused and dazzled a couple of times. Every now and then I still seem to have trouble to actually understand these settings, after a quick scan through the VCDX Enterprise Study Guide by Peter I decided to write this post and I took the time to get to the bottom of it. I needed this settled once and for all, especially now I start to focus more on BC/DR.

According to the San Config Guide(vi3_35_25_san_cfg.pdf) there are three states:

  1. EnableResignature=0, DisallowSnapshotLUN=1 (default)
    In this state, you cannot bring snapshots or replicas of VMFS volumes by the array into the ESX Server host regardless of whether or not the ESX Server has access to the original LUN. LUNs formatted with VMFS must have the same ID for each ESX Server host.
  2. EnableResignature=1, (DisallowSnapshotLUN is not relevant)
    In this state, you can safely bring snapshots or replicas of VMFS volumes into the same servers as the original and they are automatically resignatured.
  3. EnableResignature=0, DisallowSnapshotLUN=0 (This is similar to ESX Server 2.x behavior.)
    In this state, the ESX Server assumes that it sees only one replica or snapshot of a given LUN and never tries to resignature. This is ideal in a DR scenario where you are bringing a replica of a LUN to a new cluster of ESX Servers, possibly on another site that does not have access to the source LUN. In such a case, the ESX Server uses the replica as if it is the original.

The advanced LVM setting EnableResignature is used for resignaturing a VMFS volume that has been detected with a different LUN ID. So what does the LUN ID has to do with the VMFS volume? The LUN ID is stored in the LVM Header of the volume. The LUN ID is used to check if it’s the same LUN that’s being (re)discovered or a copy of the LUN that’s being presented with a different ID. If this is the case the VMFS volume needs to be resignatured, in other words the UUID will be renewed and the LUN ID will be updated in the LVM header.

UUID, what’s that? Chad Sakac from EMC described it as follows in his post on VMFS resignaturing:

It’s a VMware generated number – the LVM signature aka the UUID (it’s a long hexadecimal number designed to be unique). The signature itself has little to with anything presented by the storage subsystem (Host LUN ID, SCSI device type), but a change in either will cause a VMFS volume to get resigned (the ESX server says “hey I used to have a LUN with this signature, but it’s parameters were different, so I better resign this”).

Like Chad says the UUID has little to do with anything presented by the storage subsystem. A VMFS volume ID aka UUID looks like this:

42263200-74382e04-b9bf-009c06010000

1st part – The COS Time when the file-system was created or re-signatured
2nd part – The TSC Time; an internal time stamp counter kept by the CPU
3rd part – A random number
4th part – The Mac Address of the COS NIC

Like I said before, and this is a common misconception so I will say it again, the LUN ID and the Storage System product ID are stored in the LVM header and not the actual UUID itself. Not that it really matters for the way the process works though.

I think that makes it clear when to use EnableResignature and when not to use it. Use it when you want to access VMFS volumes of which the LUN ID changed for whatever reason. For instance a fail over to a DR Site with different LUN numbering or SAN upgrades which caused changes in LUN numbering.

That leaves DisallowSnapshotLun. I had a hard time figuring out when to set it to “0” and when to leave it at the default setting “1”. But found the following in a VMworld Europe 2008 presentation:

DisallowSnapshotLun: Should be set to “0” if SCSI Inquiry string differs between the two Array’s in order to allow access to datastores.

I googled for “SCSI Inquiry” and I found the following in a document by HP:

The storage system product ID retrieved from the SCSI Inquiry string (Example: HSV210)

In other words, when you’ve got an HP EVA 4000 and an HP EVA 8000 which are mirrored you need to set DisallowSnapshotLun to 0, when a fail-over has occurred. The SCSI Inquiry string would differ because the controllers would be of a different model. (The SCSI Inquiry string also contains the LUN ID by the way.)

When both sites are exactly the same, including LUN ID’s, you don’t need to change this setting. Leave it set to 1. Be absolutely sure that when you set DisallowSnapshotLun to 0 that there’s only 1 “version” of the VMFS volume presented to the host. If for some reason both are presented data corruption can and probably will occur. If you need to present both LUNs at the same time, use EnableResignature instead of DisallowSnapshotLun.

Depending on the way your environment is setup and the method you chose to re-enable a set of LUNs you may need to re-register your VM’s. The only way to avoid this is to use DisallowSnapshotLun and pre-register all VM’s on the secondary VirtualCenter server or use just one VirtualCenter server.

Re-registering can be done with a couple of lines of script on just one ESX box:

for i in `find /vmfs/volumes/ -name "*.vmx" `
do
echo "Registering VM $i"
vmware-cmd -s register $i
done

You can change the EnableResignature or DisallowSnapshotLun setting as follows:

open vCenter
Click on a host
Click on “Configurations” tab
Click on “Advanced Settings”
Go to “LVM”
Change appropriate setting
Click “Ok”
Rescan your HBA’s (Storage Adapters, Rescan)

It’s also possible to use the command line to enable DisallowSnapshotLun or EnableResignature:

echo 0 > /proc/vmware/config/LVM/DisallowSnapshotLUN
echo 1 > /proc/vmware/config/LVM/EnableResignature

I do want to stress that setting the options should always be used temporarily considering the impact these changes can have! When you set any of both options reset them to the default.     The big question still remains, would I prefer resignaturing my VMFS volumes or setting “DisallowSnapshotLun” to “0” to be able to access the volumes? Well the answer is:”It depends”. It heavily depends on the type of setup you have, I can’t answer this question without knowing the background of an environment. The safest method definitely is Resignaturing.

Before you decide read this post again and read the articles/pdf’s in the links below that I used as a reference:

Updates for the VMFS volume resignaturing discussion
HP disaster tolerant solutions using Continuous Access for HP EVA in a VI 3 environment
Fibre Channel SAN Configuration Guide
VMFS Resignaturing by Chad Sakac

Reader Interactions

Comments

  1. I’ve run into one nightmare with EnableResignature and/or DisallowSnapshotLUN in my prod environment and IMO one experience with this is enough for anyone.

    I had not tweaked these settings previously but I was forced to get intimate with them when for some strange reason, clustered ESX hosts started seeing existing SAN LUNs with new names, basically the same correct name, but with the infamous (1) behind it. So basically HOSTA sees a LUN as LUN1 and HOSTB sees the same LUN as LUN1(1).

    Getting things settled down was a pain in the ass and I was sweating it for a little while.

  2. BTW, very nice post Duncan. I hope that others would verify that I know what kind of time/energy/knowledge it takes to write quality blog post like this.

  3. I’ll test it this saturday, one of our customer will migrate his emc storage, i’ll be there to monitor the vmware infrastructure.
    Let me explain what I think it will occur (yours comments are welcome)
    1)shutdown every vm
    2)shutdown every esx
    3)migrate storage (using sancopy)
    4)mask or remove old storage, I don’t want esx to see them (point 3 and 4 will be achieved by the storage team)
    5)start one esx (I hope it will mark copied lun as snapshot)
    6)set EnableResignature=1 then rescan lun (I hope copied lun won’t be marked as snapshot)
    7)start all esx
    8)reregister all vm (I think every vm will be orphaned or disconnected, old vmx point to old lun)

    That’s all

    Do you think I’ll be happy saturday evening? 🙂

  5. I ran in to this, when I was unplugged and replugging in multiple sata drives into different ports on my unsupported ESXi box. The test box would see the drive but not the datatstore, therefore I had to enable Resignature to get it to see it again.

    I linked to this post on my blog.

    Thanks again.

    Roger L

    http://rogerlunditblog.blogspot.com/

  6. I’ve also seen this when switching between clusters and EMC SAN’s. One LUN and machines moved to a different cluster and VirtualCenter Instance. Changing the EnableResignature setting worked a treat.

    Also had it with some DAS and Dell servers (Direct Attached Storage) after a SCSI card firmware update and once again this setting enabled read/write on the LUN. Heart in mouth stuff certainly if you’ve never heard of it before.

    Paul

  7. very intersting article
    we have come across this with a client recently using EMC recoverpoint(different SAN’s, sites and clusters) and EMC best practice dictates to re signature, however if you have a 2nd VMDK on another LUN the reregistering process is very timeconsuming due to datastore having the snap-LUNNAME, however with Disallow snapshot would be far quicker as the datastore name will be the same as what teh vmx refers to
    If you have seperate sites, with seperate SAN’s and ESX clusters then is the Disallow the best method ?

  8. Like I said, it depends. How many VC’s do you guys have? If you have just 1 VC than there’s no direct real need to reregister.

  9. I have used resignaturing once when several “stand-alone” ESX servers were grouped together in a cluster. Each ESX had its own, single lun and unfortunately some where presented with identical lun numbers in the SAN config. We had to re-number some luns to be able to present them to all the ESX servers. This caused some luns to be seen as snapshot luns.
    I found the follwing article of much use: http://tinyurl.com/3qsldy

    It describes all the steps that need to be taken.

  10. The misconception of LUN IDs being stored in the UUID may have something to do with the following passage from Mike Laverick’s SRM book;

    “UUIDs are generated by using three core variables including date, time and LUN number in order to guarantee that the UUID value be absolutely unique”.

  11. Hi,
    I’ve a big problem with my two datacenter. I’ve a cluster with 8 ESX server, 4 in datacenter A and other 4 in datacenter B, I’ve 2 SAN in PPRC (remote replica). All the host ESX see only one SAN in datacenter A and PPRC replicates date to the second SAN in datacenter B. Now we have to shutdown datacenter A and reconfigure zoning to show the disk of the second SAN to the survived ESX in datacenter B. The SAN are different type. With PPRC, I’m sure that signature are replicated, in your opinion what is the best configuration for me? Can I keep EnableResignature=0, DisallowSnapshotLUN=1 (default)
    or I’ve to change it in what?

    Many thanks for your suggestions.

  12. different type of san: resignature always! (enableresignature=1) You will however need to reregister the VMs though!

  13. Many thanks for your quickly answer, but when I failback the other 4 servers will need to resign the disks?

  16. Hi Duncan,
    I’ve a new question, if I remove a Datastore from a server in a cluster, there is some problem for other servers? Thare is some special procedure to follow or only migrate all virtual machines?

    Many thanks.

  17. I’ve got a test ESXi 4 box that had three LUNs that are being presented to it from a QNAP NAS/iSCSI storage device. After a recent power and subsequent UPS failure, ESXi no longer wants to mount these LUNs unless I resignature them, which is less than ideal since I only want the originals to show up as datastores. When I try to add them WITHOUT resig, I get an error “Cannot change the host configuration.”

    I get this error both with the vSphere client or by using “vicfg-volume -M” through vMA.

    Any ideas?

  18. Nevermind – just answered my own question. Apparently you can’t do this through the vSphere client when connected through vCenter. Once I connected my client directly to the ESXi host, I was able to re-add the volumes.

  19. Hi Justin Moore.
    I have same problem you did. How do you connect the vSphere client directly to the ESXi host and not thru vCenter ? Ho connect to the ESXi hosts ip and I get “Cannot change the host configuration” message. what’s wrong.

  21. >You can change the EnableResignature or >DisallowSnapshotLun setting as follows:
    >open vCenter
    >Click on a host
    >Click on “Configurations” tab
    >Click on “Advanced Settings”
    >Go to “LVM”
    Hi,

    I can’t find the LVM option in advanced settings? I am looking at ESX4.1U1. Is something I am missing?

    I am trying to find a way to set “EnableResignature” using vSphere client GUI. Please let know. Thanks in advance.