• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

VSAN Design Consideration: Booting ESXi from USB with VSAN?

Duncan Epping · Mar 10, 2014 ·

** Update, as of November 21st we also support SD/USB boot with higher memory configurations when core dump partition is increased. Also read this KB article on the topic of increasing the ESXi diagnostic partition size **

One thing most probably won’t realize is that there is a design consideration with VSAN when it comes to installing ESXi. Many of you have probably been installing ESXi on USB or SD and this is also still supported with VSAN. There is one caveat however and this caveat is around the total number of GBs of memory in a host. The design consideration is fairly straight forward and also documented in the VSAN Design and Sizing Guide. Just to make it a bit easier to find it I copied/pasted it here for your convenience:

  • Use SD, USB, or hard disk devices as the installation media whenever ESXi hosts are configured with as much as 512GB memory. Minimum size for USB/SD card is 4GB, recommended 8GB.
  • Use a separate magnetic disk or solid-state disk as the installation device whenever ESXi hosts are configured with more than 512GB memory.

You may wonder what the reason is, the reason for this is that VSAN will use the core dump partition to store VSAN traces that can be used by VMware Global Support Services and the VMware Engineering team for root cause analysis when needed. So make sure when configuring host to keep this in mind when going above 512GB of memory.

Please note that this is what has been tested by VMware and will be supported, so this is not just any recommendation but could have impact on support!

Share it:

  • Tweet

Related

Software Defined, Storage, vSAN 1.0, 5.5, u1, virtual san, VMware, vsan, vSphere

Reader Interactions

Comments

  1. forbsy says

    10 March, 2014 at 13:39

    I understand VSAN uses part of the SSD as a write buffer. I can’t find anywhere where it explains in detail what the write buffer is. What I mean is that I understand what a write back and write through cache is. I’d like to understand how a write buffer differs from write back, and why does VSAN employ write buffer instead? Am I incorrect in assuming that with a write buffer, writes get acknowledged back to the host immediately and destaged to spinning disk asynchronously? If so, that sounds like write back. Just would like to understand the difference.

    Thanks

    • John Nicholson. says

      10 March, 2014 at 21:07

      Forbsy, Its a Write Back Cache that is mirrored to two hosts and also stored on persistent flash instead of RAM. (This is also why you want to use good enterprise class flash drives like the Intel 3700 that have capacitors built in so they tolerate power loss without loosing data).

      • forbsy says

        10 March, 2014 at 22:00

        Thanks John. That’s interesting. I thought there might have been a difference. I wonder why VMware just doesn’t refer to it as a write back cache?

      • [email protected] says

        10 March, 2014 at 22:35

        It is mirrored to multiple hosts depending on how you’ve set “failures to tolerate”.

        • forbsy says

          10 March, 2014 at 23:07

          Thanks Duncan. I guess my confusion was comparing the terminology with PernixData. As you know that also mirrors writes to multiple hosts depending on the replication policy. PernixData refers to write back and VSAN uses write buffer. It sounds from John’s reply like they are the identical thing – just different terminology.

          • forbsy says

            10 March, 2014 at 23:10

            Not talking about identical solutions :). Just identical use of how writes are handled (i.e. quick acknowledgement of write back to host, replicate write, destage write to HDD).

  2. John Nicholson. says

    10 March, 2014 at 21:09

    Thanks for the update! I think Rawlinson was getting annoyed at everyone speculating…

  3. MIchael says

    11 March, 2014 at 23:48

    In Q3 2014 the server vendors bring 32 GB and 64 GB sd cards. Today 16 GB is no problem.

  4. Chris says

    15 March, 2014 at 16:53

    Hi Duncan, what about auto deploy stateless/stateless caching, is this a good practice or recommended for VSAN ? Of course we would install a syslog and dump collector server.

    I didn’t had the chance to read the VSAN design guide yet, the answer might in it, it´s in my to do for the next weeks.

    Thansk!

  5. Michael says

    13 April, 2014 at 14:42

    64 GB sdcards : http://www.thessdreview.com/daily-news/in-the-news/samsung-releases-new-lineup-sd-micro-sd-flash-memory-cards/ or
    http://www.sandisk.com/products/memory-cards/sd/extremepro-sdxc-sdhc-uhs-1-95mbs/?capacity=64GB

Primary Sidebar

About the author

Duncan Epping is a Chief Technologist in the Office of CTO of the Cloud Platform BU at VMware. He is a VCDX (# 007), the author of the "vSAN Deep Dive", the “vSphere Clustering Technical Deep Dive” series, and the host of the "Unexplored Territory" podcast.

Upcoming Events

Feb 9th – Irish VMUG
Feb 23rd – Swiss VMUG
March 7th – Dutch VMUG
May 24th – VMUG Poland
June 1st – VMUG Belgium

Recommended Reads

Sponsors

Want to support Yellow-Bricks? Buy an advert!

Advertisements

Copyright Yellow-Bricks.com © 2023 · Log in