Yellow Bricks

by Duncan Epping

4.1

Ephemeral ports?

· ·

A couple of days ago one of my colleagues released an article about Ephemeral Ports. The article explains about how Ephemeral ports could be used as a “backup” when vCenter is down. The summary of the article is in my opinion the paragraph I quoted below.

If the inability to quickly provision a new VM or to reconnect a vNIC while vCenter Server is unavailable has kept you from considering a pure vDS network architecture, ephemeral port groups may be a suitable safety net.  You would not even need to use ephemeral port groups for production virtual networks — simply create a few to have as backups for accessing the most critical VLANs.

This started a discussion internally as the default setting is not Ephemeral but Static. So the question that this resulted in was should we define a new standard or are the “Static” port binding just as good as Ephemeral? I believe that many people are hesitant of using a pure vDS infrastructure due to the inability to make changes to the vDS when vCenter would be unavailable. This applies to both ephemeral and static however and actually leads to another point, which we won’t discuss now, vCenter resiliency. Now, from a virtual machine perspective even if vCenter is down, and Static is used as the port bindings, the virtual machine can be powered on and off. With Static all ports are pre-defined on the host level and when a virtual machine is assigned a port it can consume it. Now the difference between Ephemeral and Static is that Ephemeral allows you to assign “new ports” to new virtual nics or virtual machines. I guess the question is how often do you make changes to the network of your virtual machines when vCenter is down and what type of changes?

Seriously, do we really want to make substantial changes to our environment when our management platform is not available? I believe we shouldn’t and I also feel that Static portgroups are the way forward, they have more or less the same level of flexibility Ephemeral have and on top of that Static offers a lot of advantages from a scaling perspective!

das.failuredetection time and relationship with isolation response

· ·

I had this question coincidentally two times of the last 3 weeks and I figured that it couldn’t hurt explaining it here as well. The question on the VMTN community was as follows:

on 13 sec: a host which hears from none of the partners will ping the isolation address
on 14 sec: if no reply from isolation address it will trigger the isolation response
on 15 sec: the host will be declared dead from the remaining hosts, this will be confirmed by pinging the missing host
on 16 sec: restarts of the VMs will begin

My first question is: Do all these timings come from the das.failuredetectiontime? That is, if das.failuredetectiontime is set to e.g. 30000 (30 sec) then on the 28th second a potential isolated host will try to ping the isolation address and do the Isolation Response action at 29 second?

Or is the Isolation Response timings hardcoded and always happens at 13 sec?

My second question, if the answer is Yes on above, why is the recommendation to increase das.failuredetectiontime to 20000 if having multiple Isolation Response addresses? If the above is correct then this would make to potential isolated host to test its isolation addresses at 18th second and the restart of the VMs will begin at 21 second, but what would be the gain from this really?

To which my answer was very short fortunately:

Yes, the relationship between these timings is das.failuredetectiontime.

Increasing the das.failuredetectiontime is usually recommended when an additional das.isolationaddress is specified. the reason for this is that the “ping” and the “result of the ping” needs time and by added 5 seconds to the failure detection time you allow for this test to complete correctly. After which the isolation response could be triggered.

After having a discussion on VMTN about this and giving it some thought and bouncing my thoughts with the engineers I came to the conclusion that the recommendation to increase das.failuredetectiontime with 5 seconds when multiple isolation addresses are specified is incorrect. The sequence is always as follows regardless of the value of das.failuredetectiontime:

  • The ping will always occur at “das.failuredetectiontime -2”
  • The isolation response is always triggered at “das.failuredetectiontime -1”
  • The fail-over is always initiated at “das.failuredetectiontime +1”

The timeline in this article explains the process well.

Now, this recommendation to increase das.failuredetectiontime was probably made in times where many customers were experiencing network issues. Increasing the time decreases the chances of running in to an issue where VMs are powered down due to a network outage. Sorry about all the confusion and unclear recommendations.

New Whitepaper: VMware ESXi 4.1 Operations Guide

· ·

As part of my new role within VMware Technical Marketing I am responsible for creating collateral. Most of you have seen the series of articles about the operational differences between ESX and ESXi. After finalizing the series I transformed them into a whitepaper. I guess one thing that stood out for me while going through that process is that writing a whitepaper is substantially different than writing a blog article and even a book. I am not sure how to explain it, but a whitepaper feels less personal and more official and requires a different writing style. On top of that there are of course multiple reviews, style edits and much more. But anyway, that is not the point of this article… I just wanted to let you know that it is out there, and I hope you will enjoy reading it.

VMware ESXi Operations Guide

Learn how to perform common datacenter tasks in your ESXi environment by seeing the operational differences from the legacy ESX architecture.

Download Operations Guide

Another whitepaper I wanted to point out is the ESXi Migrations Guide. It has been written by my colleague Kyle Gleed and is an excellent start for those looking to migrate from ESX to ESXi in the near future. Not only is the whitepaper very useful, but I am also confident you will appreciate the checklists and the configuration sheet which will help with a smooth transition.

VMware ESXi Migration Guide

Learn how to plan and perform your migration to the ESXi architecture from the legacy ESX framework, with helpful checklists for organizing the steps involved.

Download Migration Guide

Download Migration Checklists

Download Host Configuration Worksheet

We are also working on automating some parts of the upgrade, and I hope to be able to publish an update on that soon.

What if you were to design your own server…

· ·

Lately I have been thinking about the future of servers and more specifically the design around servers. Servers are more and more heading towards these massive beasts with all sorts of options that many might not need, but end up paying for as they are already bolted on. On the other hand you have these massive blade chassis that will allow for 10 / 14 blades, whatever your vendor decides is a nice form factor. While thinking about that I wondered why we have the 1U and 2U servers stuffed with options and the possibility to add disks when all we actually want, in many cases, is to run ESXi as a hypervisor. Even if we want to have local disks do we really need a 2U server?

After doing some research on the internet I bumped into something which I thought was a cool concept. Although it isn’t was I envisioned, it is close enough to share with you. I haven’t seen these types of servers used for virtualization so far and I wonder why that is. There are multiple vendors with offerings like these but I wanted to point out the following two as they offer more than others in my opinion and are VMware Certified. These servers are traditionally used in HPC environments (High-performance computing), but if you look at what they offer they could be suitable for virtualization as well. They are very dense but don’t bring along the requirement to buy a full chassis if you just need 3 or 4 servers. Of course you cannot directly compare them to blade servers and chassis, but think about the possibilities for a second and I will expand on that as well in a second.

Now in this case, the Super Micro 2U Twin2 has 4 nodes. Each node has a set of 6 SAS drives to its disposal and can hold up to 192GB or RAM. On top of that it can hold 2 Intel Nehalem/Westmere CPUs and has an Infiniband 20Gbps on board. This by itself is a very cool concept, but what if we would simplify it? These servers typically have:

  • Expansion slots
  • Sata / sas controllers
    • Disks
    • CD/DVD
  • Multiple 1GbE links
  • IPMI Lan port

But do we really need all of that? Wouldn’t a fully stripped down server make more sense for a virtualized environment? Do we really need a Sata/SAS controller? Do we need a CD/DVD Drive? Do we need multiple 1Gbe links plus 20GbE Infiniband and on top of that an IPMI Lan port? What if someone would come out with a server that wasn’t geared towards HPC but to virtualization. Yes we have seen many vendors taking their traditional servers and positioning them as Virtualization Ready but are they? So what would I like to see?

Well for starters I kinda like the form factor above, but I would like to see one without those disks. In most environments there will be shared storage available so there is no need for local disks. It would be nice if they had an on-board dual SD slot, allowing for ESXi to be installed locally. So what if someone could crank out, maybe someone already did if so let me know, a configuration like this:

  • 2U “Chassis”
  • Max 4 nodes
  • Each node supporting max 2 sockets
  • Each node supporting 192GB (probably overkill)
  • Single 10GbE CNA
  • Single IPMI LAN port
  • SAS/SATA controllers

But what if we could go even more crazy like that, kinda like what Dell developed with their C5125 Microservers, what if you could host 12 Server nodes in 3u? Would that be something that you would be interested in? Yes, you might be limited to a single processor but without the requirement for a disk and lets say 96GB of memory max it should be possible. Yes I understand their would be implications to a design like that, but that is not the point right now.

I don’t design hardware or servers, but it seems to me that many options have been explored for all kinds of workloads but we haven’t reached the full potential for virtualization. Out in the field we see many people creating home labs with barebone casings, we see people running very stripped down configurations but when you walk into a random datacenter you see DL380’s, Dell R710s etc fully stocked with all bells and whistles while half of these features are not used. Wouldn’t dense and virtualization purpose built servers be nice? Seamicro created a nice solution with 512 servers in a 10 Us, but the CPUs are not powerful enough unfortunately for our purpose. Still I feel there are opportunities out their to really innovate, to lower the cost, lower the chances of failure and to ease management and maintenance!

Which server vendor out there is going to take the next step?

Home Lab expansion…

· ·

I’ve posted an overview of my homelab a while back and it changed a bit over the course of the last couple of months so I wanted to do an update of the article. Let me disclose first that Drobo was kind enough to provide me with a test-unit. Thanks Drobo!

My Workstation which runs Windows 7 with VMware Workstation on top of it. The most important change is the addition of an SSD drive. I ran two nice Seagate Cheetah 15k SAS drives in RAID-0 for a while, but started to get annoyed by the ticking sound these drives produce. It’s not a defect it is part of the mechanism, but very annoying background noise.

And another substantial change is the lab storage. I used to run on two Iomega IX4’s. Although these are very cool devices unfortunately they are “limited” to four drives and I was looking for some more capabilities to extend some of the tests I am conducting. I just received a brand new Drobo B800i with 6 x 7.2k Sata drives. Which means I have two slots left which I might just fill up with SSD for the sake of it.

  • Drobo B800i (new)
  • 6 x Western Digital 7.2k Drive

If I would give one tip though to the Drobo folks, make the dashboard available over http/https rather than a separate utility. Hopefully I can do some performance testing next week or the week after when I have some more time on my hands.