Comments on: Block sizes and growing your VMFS

By: pwallace

pwallace — Thu, 10 Dec 2009 19:09:56 +0000

I am not sure I see where Satyam’s comments help dismiss disk aligning, but I am new to this so any explanation would be appreciated.

By: Tom

Tom — Sat, 30 May 2009 16:43:38 +0000

It seems that unless one has a lot of I/O issues to think about — big companies, big apps, etc. — block size might be important. But for the majority of SMBs implementing esx/vsphere, they’re not going to have gigantic I/O issues and the block size is just one more thing to remember. Makes more sense for everyone to set up their VMs aligned. I created my gold templates on aligned C:\ drives because I was creating new ones anyway. And I align any new partitions I create…I really liked Satyam’s comment. It convinced me that I don’t have too much to worry about by going with the defaults and making the effort to align disks.

By: canalha

canalha — Sat, 30 May 2009 03:06:19 +0000

A valuable discussion, guys.

Question on top of it: won’t VMFS-3 also need a lock to allocate the sub-block?

By: Duncan

Duncan — Fri, 29 May 2009 22:45:34 +0000

This is honestly one of the best replies I ever had on my blog. Thank you very much for your insights! I really appreciate it and I know for sure that all my readers will appreciate it.

By: Satyam Vaghani

Satyam Vaghani — Fri, 29 May 2009 17:51:05 +0000

I am a VMware employee and I wrote VMFS with a few cronies, but the following is a personal opinion:

Forget about locking. Period. Yes, SCSI reservations do happen (and I am not
trying to defend that here) and there will be some minor differences in
performance, but the suggestion on the (very well written) blog post goes against the mission of VMFS, which is to
simplify storage virtualization.

Heres a counter example: if you have a
nearly full 8MB VMFS volume and a less full 1MB VMFS volume, you’ll still
encounter less IO overheads allocating blocks on a 1MB VMFS volume compared
to the 8MB volume because the resource allocator will sweat more trying to
find a free block in the nearly full volume. This is just one scenario, but my point is that there are tons of things to consider if one wants to account for overheads in a holistic manner and the VMFS engineers don’t want you to bother with these “tons” of things. Let us handle all that for you.

So in summary, blocksizes and thin provisioning should be treated
orthogonally. Since thin provisioning is an official feature, the thing for
users to know is that it will work “well” on all VMFS blocksize
configurations that we support. Thinking about reservations or # IOs the
resource manager does, queue sizes on a host vs the blocksize, etc will confuse the user with
assertions that are not valid all the time.

I like the post in that it explains blocks vs sub-blocks. It also appeals to
power users, so that’s great too. But reservation vs. thin provisioning
considerations should be academic only. I can tell you about things like
non-blocking retries, optimistic IO (not optimistic locking) and tons of
other things that we have done under the covers to make sure reservations
and thin provisioning don’t belong in the same sentence with vSphere 4. But
conversely, I challenge any user to prove that 1MB incurs a significant
overhead compared to 8MB with thin provisioning

By: Sharninder

Sharninder — Tue, 19 May 2009 06:21:21 +0000

8MB all the way. There aren’t many files less than 64kb on a vmfs anyway and the sub-blocks solve the internal fragmentation problem to a large extent. 8MB sounds like the logical choice for vsphere.

Although, I’m not sure how thin provisioned disks will behave wrt. thick disks’ in performance.

By: Duncan Epping

Duncan Epping — Sun, 17 May 2009 05:38:26 +0000

Good question and I honestly don’t know. That’s one of the reasons I asked you guys to chip in…

By: craig

craig — Sun, 17 May 2009 02:41:09 +0000

I am just little curious here, if 8MB block size should be the right setting to go for, but why the block size recommendation from VMware in the administration console, always tie up a certain block size with specify datastore size? any idea? just to clarify here

By: Duncan Epping

Duncan Epping — Thu, 14 May 2009 21:57:38 +0000

I don’t think you really understood my post… Or I am really confused about what you are trying to say.

By: jl

jl — Thu, 14 May 2009 21:10:50 +0000

we will wait to see how other people experience thin-provisioned disks performance wise and if the scsi reservations are going to make problems, just saying that vsphere has optimized locking technology is not enough, need a detailed white paper that describes why it is better.