Yellow Bricks

by Duncan Epping

Project: Massive Array of Inexpensive Servers aka MAIS

· ·

I was part of the voting committee for VMworld and one of the sessions which I voted for unfortunately did not make it. However with over 350 submissions there are always some excellent topics that don’t make it. I did feel it was worth sharing. So here’s the outline of the session and some additional info. All credits to Richard Stinton (VMware Cloud Architect, EMEA) and his Team for coming up with the concept and allowing me to publish this!

Remember Simon Gallagher’s vTARDIS project? Now for something different. This is the Massive Array of Inexpensive Servers or MAIS (pronounced MAZE). We’re going to build an array of 32 (or more) $150 servers and show the power of vSphere and vCloud Director. Using the new(ish) HP Proliant Microserver, we’re going to build a wall of vBricks!

So it wasn’t just a submission, but these guys actually started working on it. They managed to get their hands on 32 HP Microservers. That gives a total of 64 cpus, 256Gb RAM, 8Tb storage, all for $9,000. They loaded it up with the latest vSphere and vCloud Director versions and it ran great, unfortunately we will never see the results but I did want to share one more thing;

Is that cool or what?

Reader Interactions

Comments

  2. I’ve only seen those around for around US$320. What currency is the $150 in? Also…. that’s freakin’ awesome!

  5. $9000 total for hardware – that is truly commodity pricing…
    I’d be interested to know the minimum vSphere + vCloud license costs to run on the vBrick “wall” as spec’ed ?

    I suppose one of the implications of this move from ESX(i) socket pricing to per VM pricing, is that the consolidation ratios become less important (eg, we had been buying big beefy Dell R910’s with 256Gb Ram since we could get the best consolidation and bang for our socket priced hypervisors) – now that VMware is moving to per VM pricing that equation changes to put smaller hardware platforms back on equal footing.

    • because the only thing i want more than a cluster of inexpensive ESXi servers is a cluster of DVD-RW’s!

      • Put the farms idle time to use by burning dvds for the black market? Side income to offset the ‘cluster’ purchase?

    • I have one of the Microservers (which has 4 disk bays) running Nexenta Community Edition NAS appliance (from a ISB key). That is serving up RAID10 with dedup via iSCSI or NFS. And it’s F-R-E-E-E-E !

  9. Id second the question by FLECTH. Giving the current license model of vShpere this solution is costly. Then there is power, cabling, cooling, space, management, etc to consider. Now if this was done with all free and opensource solutions…. Still, probably nice for a compute grid though.

    • a fully loaded, 4 HDD, 8GB, two PCI-E cards, I cannot get the Microserver above 40 Watts, with HP 4GB 165w, ESXi 4.1 U1. as for others, great little lab boxes. Presently using FC SAN storage (a bit OTT), but moving to iSCSI/NFS with Solaris ZFS in another Microserver.

  11. Pretty cool, too bad they are AMD CPUs though. I was looking at a similar idea for my lab using whitebox mini-itx based sandy bridge boxes, those can take 16gb of ram. Interesting read though, thanks!

  12. NICE! – for some reason Im thinking of Pink Floyd’s – The Wall (of MAIS)

    • LOL. I was considering ‘Comfortably Numb’ for the backing track on the YouTube video…but then I’d be showing my age!

    • Are you using the AMD ones? How are you finding it?

      I am considering purchasing two myself, for my home lab. would be awesome if you could let me know 🙂

      Just wondering if you’re using a SW or HW raid and what discs you’re using.

      Thanks, very much,

      G.

      • The MicroServer uses an AMD “Fake RAID” controller, so if you want Hardware RAID you’ll have to use a HP Smart Array P410, it has the correct mini-SAS connector so you can remove the one from the motherboard and connect to controller. Only issue is using non TLER drives on the controller, other deep recovery drops the drives off the Array, nothing unusual. You may want to consider LSI controllers, or using NFS/iSCSI storage from ZFS on Solaris Express 11, on SSD – current project for a SSD SuperSAN with ZFS. Trying to dump expensive and costly Lab HP ProLiant Servers and FC SANs due to rising electrical costs, these run at 40 Watts each box! Our SAN shelves run at 1.4Kw at shelf, and then there the air con needed in the Summer, to cool down the room! More electric! and electric in the UK, is getting more expensive, and will be three times the cost in 15 years!

      • We’ve been testing Hyper-V, ESXi 4.1/5.0, XEN 5.5 on these boxes, they run very well, and if you in the UK, you cannot beat £100 inc 8GB RAM, with the £100 cashback! (at 40Watts!) Cleverly engineered, because you can leave an ESXi USB flash drive inside the machine running ESXi 4.1, but the USB connectors on the external box have priority over the internal, so just plugin another Hypervisor to test in the external slots, shutdown and restart, bingo new Hypervisor running! Just change the NIC if using FreeBSD, because the FreeBSD bge0, broadcom driver is buggy, and keeps reseting the NIC, so it’s no good. So I highly recommend the cheap and cheerful but very good Intel® Gigabit CT Desktop Adapter.