• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

More than 8 disks in your Virtual SAN host?

Duncan Epping · Jul 17, 2015 ·

It seems Virtual SAN is on fire… I’ve been getting emails about VSAN configurations on a daily basis from partners, VMware field and customers. More and more customers seem to be looking at deploying Virtual SAN with a high number of disks and this week I ran in to something that I wanted to share with you. I thought it was common knowledge, but I guess I was wrong. If you are planning to have more than 8 disks / devices in your Virtual SAN host then it is good to know that you will either need a second disk controller (Check the HCL!) or use a SAS Expander! This design / sizing principle is listed in the VSAN Ready Node PDF, and should also make its way to the Design / Sizing Guide soon. The following is mentioned in the VSAN Ready Node PDF:

No more than 8 disks (SSD + HDD) supported behind a single controller. In case you need to add more than 8 disks, please add an additional controller.

I met up with the product manager for the compatibility guide and asked him which SAS Expanders are supported, he mentioned that support is done per platform and that the following vendors and platforms have just been update. Note that the HCL will still need to be updated for these:

  • Dell SAS Expanders 5.5 and 6.0 hybrid. Supported on a specific configuration: H730 on 730d platform with 24 drives.
  • Lenovo SAS Expanders 5.5. Supported on a specific configuration: Lenovo 720ix, RD650 platform with 12 drives.

We also are working on support for HP and Cisco (and others over time), and you can also expect an update on those soon. For now, keep in mind that when designing / planning for Virtual SAN the number of virtual disks may mean you need to change the configuration of your host.

Related

Server 6.0, virtual san, vsan

Reader Interactions

Comments

  1. Bryan Pizzuti says

    17 July, 2015 at 20:03

    Good thing I used an R730xd then. Haven’t run into an issue yet, I’m even using the H330 instead of an H730, this way I don’t have to do the RAID0 thing.

    • robbie smith (@virtualrob) says

      18 July, 2015 at 17:20

      All dell 13th generation percs support disk passthru natively so the h730 does not require a raid0 per disk. It was the previous generation that had the limitation.

  2. Mark says

    17 July, 2015 at 20:39

    8disk max is common, also sas expander. you cannot use standard sas expander on hp server and also no 2nd controller to get devices into vsan? really?

    • Duncan Epping says

      20 July, 2015 at 10:35

      Euuh, as the post an the quote of the VSAN Ready Node states:
      “In case you need to add more than 8 disks, please add an additional controller.”
      and
      “We also are working on support for HP and Cisco ”

      More details for HP soon, but in the meanwhile a second controller should be no problem at all.

  3. Drew says

    17 July, 2015 at 21:00

    I’ve been running Dell 730xd Virtual SAN clusters with 18 drives per server (3 disk groups) successfully for a while now. There was a struggle at first where I had to run in RAID 0 before they released new drivers for the PERC H730P controllers. But have been running in pass-through mode for over 6 months now. I may not be pushing them as hard as some people might but for our usage they have been working well.

    • Braf says

      18 July, 2015 at 02:43

      So Drew, you’re running 3 H730p controllers in your setup?

      • Drew says

        20 July, 2015 at 17:46

        No, I am running 1 h730p in each server.

    • Duncan Epping says

      22 July, 2015 at 10:15

      As of last week supported up to 24 drives with a SAS Expander!

  4. Brad says

    18 July, 2015 at 02:46

    Bad timing that I just saw this now. Our R730xds just came in. One H730, Two SSDs and 8 SAS drives each.

    • Duncan Epping says

      22 July, 2015 at 10:15

      Supported with the R730XD as of last week, so you are good.

  5. Kenneth Slish says

    18 July, 2015 at 05:10

    Is this a recent development? I’m confused. I literally just racked 4 Dell R730’s for a VDI cluster and had them spec’d exactly like the “Value Configuration” on page 34 of their (Dell’s) VSAN whitepaper at:

    http://en.community.dell.com/cfs-file/__key/telligent-evolution-components-attachments/13-4491-00-00-20-43-81-92/VMware-Virtual-SAN-for-ESXi-5-5-_2D00_-v.6.5.5.pdf

    2 disk groups. 1 SSD and 6 300GB 15k drives per group for a total of 14 drives. Just 1 H730P mini controller just like in that document.

    Figured I couldn’t go wrong following a tested design. I can’t add another disk controller due to the 2 nVidia GRID cards, a Teradici card, 10Gig network card and FC card connecting to our legacy storage SAN.

    • Jon says

      18 July, 2015 at 08:02

      Hmm… Sounds like something worth knowing. If you are running 14 total SAS disks and two of them being your SSD performance tier, their could be bandwidth contention… maybe. This may not be the case when utilizing PCIe as your performance tier. Just a thought

    • Duncan Epping says

      22 July, 2015 at 10:18

      Which controller did you use? If you used the H730 then you are good, supported as of last week with up to 24 drives

  6. Myles Gray says

    19 July, 2015 at 17:02

    Duncan, We specced our VSAN cluster off the Dell 12G (R720XD) generation, and looking at that document it states that with 8 HDD + 2 SSD we only need 1x H710p.

    However I believe the R720XD backplane has a built in SAS Expander. (2x SFF-8087 cables for 12+2 drives).

    We have populated the box with 12+2 as of course this makes most commercial sense.

    Can you clarify on whether the Dell backplanes are supported SAS Expanders?

    • Duncan Epping says

      22 July, 2015 at 10:19

      http://partnerweb.vmware.com/programs/vsan/Virtual%20SAN%20Ready%20Nodes.pdf

      According to the note in this paper you should be good.

  7. Paul. says

    7 August, 2015 at 17:21

    uuuurgh I have just installed 10 x Cisco UCS C240 M3S hosts, which have one LSI 6G MegaRAID 9271-8i cards… each host has 2 x 400GB SAS SSD & 12 X 900GB 6G SAS spindles.. This kit was purchased before my arrival at the company.. so we need additional controllers right Duncan? Cheers @pshearduk

    • Duncan says

      7 August, 2015 at 18:13

      Good question Paul, not sure about this one. I will ask one of our PMs.

      • Paul. says

        8 August, 2015 at 11:55

        Much appreciated Duncan.

  8. Paul. says

    7 August, 2015 at 17:22

    These hosts are “VSAN ready nodes” too btw

  9. Paul. says

    10 August, 2015 at 13:17

    A quick update.. If I see the 14 disks (12 x SAS, 2 x SSD) via the LSI MegaRAID BIOS, does this mean that I already have a SAS expander installed? or do I still need the 2nd LSI card?

    Thanks
    Paul.

    • Duncan Epping says

      10 August, 2015 at 17:52

      Your scenario is fine, you are supported. It is with new 12G controllers that these restrictions apply.

    • Brad says

      11 August, 2015 at 20:51

      Yes, just to confirm Duncan’s above comments. We have our VSAN up and in the testing phase housed in 4 R730xds, single H730 in each, Two disk groups of 4 SAS10k and 1 SSD per group.
      No problems yet.

  10. Bob says

    15 December, 2015 at 05:54

    Duncan – so you are saying the 12G controllers, like the Cisco 12G SAS Modular Raid Controller, max out at 8 drives per controller?

    Thanks,
    Bob

    • Duncan Epping says

      15 December, 2015 at 11:17

      Yes that is what I am saying.

Primary Sidebar

About the author

Duncan Epping is a Chief Technologist in the Office of CTO of the Cloud Platform BU at VMware. He is a VCDX (# 007), the author of the "vSAN Deep Dive", the “vSphere Clustering Technical Deep Dive” series, and the host of the "Unexplored Territory" podcast.

Upcoming Events

May 24th – VMUG Poland
June 1st – VMUG Belgium

Recommended Reads

Sponsors

Want to support Yellow-Bricks? Buy an advert!

Advertisements

Copyright Yellow-Bricks.com © 2023 · Log in