Yellow Bricks

by Duncan Epping

Queuedepth, how and when

· ·

So you’ve heard this probably from a few dozens of people by now when you don’t hit the expected SAN performance: Set your queuedepth to a larger size.

So how do you set this queuedepth? Find out for which module you’ll need to set this option:

vmkload_mod -l | grep qla

Now set it to a depth of 64 for module qla2300_707

esxcfg-module -s ql2xmaxqdepth=64 qla2300_707
esxcfg-boot –b

So now you’ve set the queue depth to 64 for your HBA cards, but why? Well I hope the answer is:”because I monitored my system with esxtop and I noticed that the “QUED” value was high”.

So there’s your when. You’ll need to set this setting if you notice a high “QUED” value in esxtop. Take a look at the following example I borrowed from a great blog on this subject:

As you can see in the example, the “ACTV” has a value of 32. Indeed 32 active commands cause that’s the default queue depth for qlogic cards. And 31 outstanding commands, in other words if we bump up the queue depth to 64 than all the commands should be processed instead of queued in the VMkernel.

What will this result in?

Reader Interactions

Comments

  1. Brent Ozar says

    You should also check your fiber switch metrics for errors, because you’re probably causing collisions. Cranking up the queue depth too far can mean cranking up the error numbers.

  2. tgreat says

    Hi, what can I do if there is no data in columns ACTV/QUED/%USD/LOAD (there is a “-” sign there instead) ?

  4. AlexB says

    @tgreat: Have the same Problem

    @ Duncan: Did not work for the colums ACTV/QUED/%USD/LOAD …

    Any solution?

  5. Duncan Epping says

    I am going to look into this, see if I can find any related problems… can’t promise anything though cause I’m not able to reproduce this it’s hard to troubleshoot.

  6. AlexB says

    Got it!

    Open esxtop
    Press “d” for Disk Statistics
    Press “v” to display the storage system information per virtual machine
    or
    Press “u” to display the information per storage device.

    Found the information on the technical note from VMware about “Perfomance Analysis Methods”

  7. JasonD says

    I assume this problem also applies to ESXi? How would you increase the q-len on that platform?