I am working on a large ESXi deployment and thought I would start writing down some of the lessons learned. I will try to create a single post every week, if I can find the time that is.
Scratch!
There are two things that stood out the couple of days, on a technical level, when I was reading the ESXi installable documentation:
One of the things that used to be a requirement was the Scratch Partition. It appears that with vSphere this requirement has been removed:
During the autoconfiguration phase, a 4GB VFAT scratch partition is created if the partition is not present on another disk. When ESXi boots, the system tries to find a suitable partition on a local disk to create a scratch partition. The scratch partition is not required.
Of course this does not necessarily mean that you do not need one as explained in the second part of the paragraph:
It is used to store vm-support output, which you need when you create a support bundle. If the scratch partition is not present, vm-support output is stored in a ramdisk. This might be problematic in low-memory situations, but is not critical.
So the question remains what would my recommendation be? The answer is it depends, yes I know the easy way out. But when you have enough RAM on a host and from experience know that usually you only create support dumps on hosts which are in maintenance mode then don’t worry about it and don’t create it. However if you feel there is a need to create vm-support dumps while running production make sure there is a scratch partition with enough free space available.
Support
Yes ESXi is fully supported but there are some restrictions:
- Boot from FC SAN – Experimental Support
- Stateless PXE Boot – Experimental Support
Now what does “experimental support” mean? According to the VMware website it means the following:
VMware includes certain “experimental features” in some of our product releases. These features are there for you to test and experiment with. VMware does not expect these features to be used in a production environment. However, if you do encounter any issues with an “experimental feature”, VMware is interested in any feedback you are willing to share. Please submit a support request through the normal access methods. VMware cannot, however, commit to troubleshoot, provide workarounds or provide fixes for these “experimental features”.
So does that mean that in the case of stateless the booting process is experimental? Or the installation process in the case of boot from FC SAN?
No it does not. Everything related to ESXi is “experimental”. So what does this mean? Imagine you are facing serious storage issues and you just called VMware. VMware analyzes your environment and notices that it’s a PXE booted environment, they will more than likely give your support call a lower priority. Not only a lower priority but the support is “best effort”, no guarantees.
ESXi is the future, and the stateless hypervisor will be eventually the way to go!
This looks like a promising and awesome series of posts. I can’t wait for the next article.
Hany
Hey there Duncan – We were just talking about this on Twitter. I would be interested in how to align the vm’s easily. Up until now I have been using NetApp’s mbrscan/mbralign tools. How do you plan to address that? The manual way is painful IMO.
Thank you and I look forward to the series!
Nick Triantos wrote a procedure for NFS environments to do this with mbralign: http://netapptips.com/2009/10/02/esxi-mbrscanmbralign/
I haven’t seen one for ESXi in combination with NFS. But on the otherhand I hardly have any customer who worry about alignment as performance gain is usually minimal and mostly only for a very specific subset of their estate. Do you really do this by default?
Scratch space is also very likely needed when installing patch bundles. When using the Remote CLI to install the latest v4.0 patch bundle it will likely fail without scratch space defined.
@Aaron, you don’t have to worry about alignment with Windows 2008 and up cause it’s done by default by the installer.
@Duncan, boot from SAN experimental feature, that’s what keeping us away from ESXi at the moment… Do you know when this feature will be available for production environment?
Rgds,
Didier
@PiroNet – Agreed, My alignments are only 2003 based. Although if you upgrade a vm from 2003 to 2008 it will still be unaligned. Only fresh 2008 is aligned.
@Duncan – Yes, I always align any P2V machines because they will come in unaligned and I align any templates I create so future deployments from templates will be aligned. I also teach the custoemr how to do the alignments using the NetApp Tools. Probably 90% of what I do we are attached to NetApp storage and they are very picky about it. If a customer opens a NetApp support case against something my company has installed, the first thing NetApp will do is pull a remote report from the storage and that is the first thing they check and will tell the customer they need to align before going further. I have seen as much as a 20% decrease in performance without alignments in heavy I/O customers. If NetApp tells a cusomer we didn’t do an install correctly, the customer will come back wanting to know why our install (that they paid for) wasn’t done correctly. I may not happen in all accounts but I need to cover my butt and make sure it NEVER happens since they are paying for it.
As an aside, I will be doing my first ESXi install in January but it is NFS and customer is heavy Linux shop. We’re gonna stand up a Linux client to attach to the NFS shares for the alignments.
At this time I wouldn’t sell ESXi into a non-Linux shop or non-NFS shop for this very reason. Again, I’m attaching to NetApp almost exclusively and the risk of getting “caught” is too high because I know they’ll check.
I’m with you Aaron, I always mbralign everything I can simply because I know it’s a best practice and I don’t want anyone coming back and saying I didn’t do the whole job. I do a lot of work in Linux environments, which makes it a little more touchy, but just getting the templates aligned correctly helps for the customers future deployments.