• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

Project: Massive Array of Inexpensive Servers aka MAIS

Duncan Epping · Jun 17, 2011 ·

I was part of the voting committee for VMworld and one of the sessions which I voted for unfortunately did not make it. However with over 350 submissions there are always some excellent topics that don’t make it. I did feel it was worth sharing. So here’s the outline of the session and some additional info. All credits to Richard Stinton (VMware Cloud Architect, EMEA) and his Team for coming up with the concept and allowing me to publish this!

Remember Simon Gallagher’s vTARDIS project? Now for something different. This is the Massive Array of Inexpensive Servers or MAIS (pronounced MAZE). We’re going to build an array of 32 (or more) $150 servers and show the power of vSphere and vCloud Director. Using the new(ish) HP Proliant Microserver, we’re going to build a wall of vBricks!

So it wasn’t just a submission, but these guys actually started working on it. They managed to get their hands on 32 HP Microservers. That gives a total of 64 cpus, 256Gb RAM, 8Tb storage, all for $9,000. They loaded it up with the latest vSphere and vCloud Director versions and it ran great, unfortunately we will never see the results but I did want to share one more thing;

Is that cool or what?

Share it:

  • Tweet

Related

Server mais, vmworld

Reader Interactions

Comments

  1. Brandon says

    17 June, 2011 at 22:42

    DAMN IT. That would have been really neat to see some results on.

  2. alexanderjn says

    17 June, 2011 at 22:47

    I’ve only seen those around for around US$320. What currency is the $150 in? Also…. that’s freakin’ awesome!

    • duncan says

      18 June, 2011 at 10:30

      In the UK they have these with rebate every once in a while, makes them dirt cheap.

    • level380 says

      21 June, 2011 at 00:25

      Its in pounds, so 150 British pounds = 242.73 U.S. dollars

  3. Jason Boche says

    17 June, 2011 at 23:15

    Chuck Norris built the same thing, but with 33 servers all in the same vSphere cluster.

  4. anon says

    17 June, 2011 at 23:32

    oh man – i voted for that :(. Would’ve been killer!

  5. Fletch says

    17 June, 2011 at 23:42

    $9000 total for hardware – that is truly commodity pricing…
    I’d be interested to know the minimum vSphere + vCloud license costs to run on the vBrick “wall” as spec’ed ?

    I suppose one of the implications of this move from ESX(i) socket pricing to per VM pricing, is that the consolidation ratios become less important (eg, we had been buying big beefy Dell R910’s with 256Gb Ram since we could get the best consolidation and bang for our socket priced hypervisors) – now that VMware is moving to per VM pricing that equation changes to put smaller hardware platforms back on equal footing.

    • Ceri Davies says

      18 June, 2011 at 08:43

      While VMware moving to per-VM pricing isn’t a surprise, I think I’ve failed to see them actually state that anywhere. Is there a reference please?

      • Ceri Davies says

        18 June, 2011 at 08:49

        For example, the current statement says that they aren’t changing: http://www.vmware.com/support/licensing/per-vm/

    • Ceri Davies says

      18 June, 2011 at 08:44

      “256Gb RAM” ? How much did you really mean?

      • duncan says

        18 June, 2011 at 10:37

        These boxes can only carry 8GB unfortunately.

        • Richard Stinton says

          20 June, 2011 at 12:27

          32 servers each with 8Gb….

        • krauthead says

          23 June, 2011 at 15:30

          They can carry up to 1TB of RAM! (1TB Memory (64x16GB))

    • dgibbons says

      20 June, 2011 at 17:58

      Yeah I’ve love to see that $9000 price refactored with licensing fees included.

  6. Andrew Hancock says

    18 June, 2011 at 23:47

    Yes, these are very cheap currently £200 with £100 cashback

  7. Andrew Hancock says

    18 June, 2011 at 23:49

    or you can get a fully loaded, 8GB RAM, DVD/RW, ESXi already installed on USB for £300 less £100 cashback from HP.

    https://www.serversplus.com/servers/server_bundles/633724-421%23esx

    • John says

      20 June, 2011 at 22:37

      because the only thing i want more than a cluster of inexpensive ESXi servers is a cluster of DVD-RW’s!

      • level380 says

        21 June, 2011 at 00:27

        Put the farms idle time to use by burning dvds for the black market? Side income to offset the ‘cluster’ purchase?

  8. D. says

    19 June, 2011 at 21:27

    What about common storage?

    • Richard Stinton says

      21 June, 2011 at 10:35

      I have one of the Microservers (which has 4 disk bays) running Nexenta Community Edition NAS appliance (from a ISB key). That is serving up RAID10 with dedup via iSCSI or NFS. And it’s F-R-E-E-E-E !

  9. thehyperadvisor says

    20 June, 2011 at 04:11

    Id second the question by FLECTH. Giving the current license model of vShpere this solution is costly. Then there is power, cabling, cooling, space, management, etc to consider. Now if this was done with all free and opensource solutions…. Still, probably nice for a compute grid though.

    • Andrew Hancock says

      18 August, 2011 at 19:33

      a fully loaded, 4 HDD, 8GB, two PCI-E cards, I cannot get the Microserver above 40 Watts, with HP 4GB 165w, ESXi 4.1 U1. as for others, great little lab boxes. Presently using FC SAN storage (a bit OTT), but moving to iSCSI/NFS with Solaris ZFS in another Microserver.

  10. john says

    20 June, 2011 at 15:05

    Is this some thing similar to V-block form emc-cisco-vmware

  11. Ben Thomas says

    20 June, 2011 at 16:32

    Pretty cool, too bad they are AMD CPUs though. I was looking at a similar idea for my lab using whitebox mini-itx based sandy bridge boxes, those can take 16gb of ram. Interesting read though, thanks!

    • Duncan says

      20 June, 2011 at 20:18

      Which motherboard are you looking at?

      • Ben Thomas says

        25 June, 2011 at 16:37

        I was thinking of using this one (its intel, figured it had the
        highest chance of working correctly) :

        http://www.newegg.com/Product/Product.aspx?Item=N82E16813121507

        Throw in a dual port NIC like this:

        http://www.newegg.com/Product/Product.aspx?Item=N82E16833106014

        and it would make a pretty sweet little whitebox. I started with
        2x4gb dimms since the 8gb dimms are just starting to arrive and are
        pretty pricy. Couple that with a small case and you would be good to
        go for about 600 with a sandy bridge i5 and 750 for an i7. I am
        planning to build a test one here in a bit (I am trying to wait out
        the 8gb dimms) but I think 2 or 3 of those booting from USB with some
        SOHO storage (or even an atom box with some HW raid) would be a great
        little lab.

  12. Paul Adam says

    20 June, 2011 at 22:14

    NICE! – for some reason Im thinking of Pink Floyd’s – The Wall (of MAIS)

    • Richard Stinton says

      21 June, 2011 at 10:36

      LOL. I was considering ‘Comfortably Numb’ for the backing track on the YouTube video…but then I’d be showing my age!

  13. Max says

    21 June, 2011 at 15:59

    I suggest to use this new mainboard very very cool
    http://www.bcmcom.com/bcm_product_mx67qm.htm

    • Duncan Epping says

      21 June, 2011 at 18:54

      So why that one Max?

  14. John Dias says

    22 June, 2011 at 20:44

    Love this! I own two HP Microservers for my own home lab.

    • Gabi says

      24 June, 2011 at 12:10

      Are you using the AMD ones? How are you finding it?

      I am considering purchasing two myself, for my home lab. would be awesome if you could let me know 🙂

      Just wondering if you’re using a SW or HW raid and what discs you’re using.

      Thanks, very much,

      G.

      • Andrew Hancock says

        18 August, 2011 at 19:43

        The MicroServer uses an AMD “Fake RAID” controller, so if you want Hardware RAID you’ll have to use a HP Smart Array P410, it has the correct mini-SAS connector so you can remove the one from the motherboard and connect to controller. Only issue is using non TLER drives on the controller, other deep recovery drops the drives off the Array, nothing unusual. You may want to consider LSI controllers, or using NFS/iSCSI storage from ZFS on Solaris Express 11, on SSD – current project for a SSD SuperSAN with ZFS. Trying to dump expensive and costly Lab HP ProLiant Servers and FC SANs due to rising electrical costs, these run at 40 Watts each box! Our SAN shelves run at 1.4Kw at shelf, and then there the air con needed in the Summer, to cool down the room! More electric! and electric in the UK, is getting more expensive, and will be three times the cost in 15 years!

      • Andrew Hancock says

        18 August, 2011 at 19:52

        We’ve been testing Hyper-V, ESXi 4.1/5.0, XEN 5.5 on these boxes, they run very well, and if you in the UK, you cannot beat £100 inc 8GB RAM, with the £100 cashback! (at 40Watts!) Cleverly engineered, because you can leave an ESXi USB flash drive inside the machine running ESXi 4.1, but the USB connectors on the external box have priority over the internal, so just plugin another Hypervisor to test in the external slots, shutdown and restart, bingo new Hypervisor running! Just change the NIC if using FreeBSD, because the FreeBSD bge0, broadcom driver is buggy, and keeps reseting the NIC, so it’s no good. So I highly recommend the cheap and cheerful but very good Intel® Gigabit CT Desktop Adapter.

Primary Sidebar

About the author

Duncan Epping is a Chief Technologist in the Office of CTO of the Cloud Platform BU at VMware. He is a VCDX (# 007) and the author of the "vSAN Deep Dive" and the “vSphere Clustering Technical Deep Dive” series.

Upcoming Events

04-Mar-21 | Polish VMUG – Roadshow
09-Mar-21 | Austria VMUG – Roadshow
16-Mar-21 | VMUG Turkey – Roadshow
18-Mar-21 | St Louis Usercon Keynote
26-Mar-21 | Hungary VMUG – Roadshow
08-Apr-21 | VMUG France – Roadshow

Recommended reads

Sponsors

Want to support us? Buy an advert!

Advertisements

Copyright Yellow-Bricks.com © 2021 · Log in