When designing your VMware vSphere / VI3 environment there are so many variables you need to take into account that it is easy to get lost. Something hardly anyone seem to be taking into account when creating VMFS volumes is that the metadata will also take up a specific amount of disk space. You might think that everyone has at least 10% disk space free on a VMFS volume but this is not the case. Several of my customers have dedicated VMFS volumes for a single VMDK and noticed during the creation of a VMDK that they just lost a specific amount of MBs. Most of you guessed by now that that is due to the metadata but how much disk space will the actually metadata consume?
There’s a simple formula that can be used to calculate how much disk space the metadata will consume. This formula used to be part of the “SAN System Design and Deployment Guide” (January 2008) but seems to have been removed in the updated versions.
Approximate metadata size in MB = 500MB + ((LUN Size in GB – 1) x 0.016KB)
For a 500GB LUN this would result in the following:
500 MB + ((500 - 1) x 0.016KB) = 507,984 MB
Roughly 1% of the total disk size used for metadata
For a 1500MB LUN this would result in the following:
500 MB + ((1.5 - 1) x 0.016KB) = 500,008 MB
Roughly 33% of the total disk size used for metadata
As you can see for a large VMFS volume(500GB) the disk space taken up by the metadata is only 1% and can almost be neglected, but for a very small LUN it will consume a lot of the disk space and needs to be taken into account….
[UPDATE]: As mentioned in the comments, the formula seems to be incorrect. I’ve looked into it and it appears that this is the reason it was removed from the documentation. The current limit for metadata is 1200MB and this should be the number you should use for sizing your datastores.
dougied85 says
I work for a company that has fallen foul of this calculation – in fact we are almost certainly the reason this has now been removed from the Design & Deployment Guide.
This calculation is incorrect.
We ended up in a situation where LUNs became inaccessible because they became 100% full as a result of a bug in the VMFS3 module that filled up the space we had reserved for metadata. This is also probably where the Advanced Option for OpenWithoutJournal came from, as a custom fix we were provided by Engineering gave us that option on our 3.01 environment at the time.
I have been back and forth with VMware quite a lot over this issue, and the best I have managed to get out of them to date is that you should allow 1200MB per LUN (no matter what size of LUN) for metadata. They won’t publish this officially though, as they feel LUN sizing is too dependent on other factors for them to be able to set official minimums that covers all.
Duncan Epping says
Thanks Doug, looked into it and edited my article.