Yellow Bricks

by Duncan Epping

vSphere 5.1 Clustering Deep Dive promotion & major milestone

· ·

This week when looking at the sales numbers of the vSphere Clustering Deep Dive series and Frank and I noticed that we hit a major milestone! In September 2014 we passed the 45000 copies distributed of the vSphere Clustering Deep Dive. Frank and I never ever expected this or even dared to dream to hit this milestone.

When we first started writing the 4.1 book we had discussions around what to expect from a sales point of view and I recall having a discussion with Frank around the sales number, Frank said he would be happy with 100 and I said well 400 would be nice. Needless to say we reset our expectations many times since then… We didn’t really follow it closely in the last 12-18 months, and as today we were discussing a potential update of the book we figured it was time to look at the numbers again just to get an idea. 45000 copies distributed (ebook + printed) is just remarkable, and we are very humbled, baffled and honoured!

We’ve noticed that the ebook is still very popular, and decided to do a promo. As of Monday the 13th of October the 5.1 ebook (kindle) will be available for only $ 0.99 for 72 hours, then after 72 hours the price will go up to $ 3.99 and then after 72 hours it will be back to the normal price. Make sure to get it while it is low priced!

You can pick it up here on Amazon.com! The only other kindle store we could open the promotion up for was amazon.co.uk, so that is also an option.

Project Fargo aka VMFork – What is it?

· ·

I have seen various people talking about Project Fargo (also known as VM Fork or Instant Clone) and what struck me is that many are under the impression that Project Fargo is the result of the CloudVolumes acquisition. Lets set that straight first, Project Fargo is not based on any technology developed by the CloudVolumes team. Project Fargo has been developed in house and as far as I can tell is an implementation of Snowflock (University of Toronto / Carnegie Mellon University), although I know that in house they have been looking at techniques like these for a long time. Okay, now that we have that out of the way, what is Project Fargo?

Simply said: Project Fargo is a solution that enables you to rapidly clone a running VM. When I say “rapidly clone”, I mean RAPIDLY… Within seconds. Yes, that is extremely fast for a running VM. What should be noted here of course is the fact that it is not a full clone. I guess this is where the “VMFork” name comes in to play, the “parent” virtual machine is quiesced and forked and a “child” VM is born. This child VM is leveraging the disk and memory of the parent (for reads), this is why it is so extremely fast to instantiate… as I said literally seconds, as it “only” needs to create empty delta files, create a VMX and instantiate the process, and do some networking magic as you do not want to have VMs popping up on the network with the same MAC address. Note that the child VM starts where the parent VM left off, so there is no boot process it is instant on! (just like you suspend and resume it) I can’t reveal too much around how this works, yet, but you can imagine that a technique like “fast suspend resume” (FSR), which is the corner stone of features like Storage vMotion, is leveraged.

The question then arises, what if the child wants to write data to memory or disk? This is where the “copy on write” technique comes in to play. Of course the child won’t be allowed to over write shared memory pages (or disk for that matter) and as such a new page will be allocated. For those having a hard time visualizing it, note that this is a conceptual diagram and not how it actually is implemented, I should have maybe drawn the different layers but it would make it too complex. In this scenario you see a single parent with a single child, but you can imagine there could also be 10 child VMs or more, you can see how efficient that would be in terms of resource sharing! And even for the pages which would be unique compared to the parent, if you clone many similar VMs there is a significant chance that TPS will be able to collapse those even! One thing to point out here is that the parent VM is quiesced, in other words it’s sole purpose is allowing for the quick creation of child VMs.

project fargo

Cool piece of technology I agree, but what would the use case be? Well there are multiple use cases, and those who will be attending VMworld should definitely visit the sessions which will discuss this topic or watch them online (SDDC3227, SDDC3281, EUC2551 etc). I think there are 2 major use cases: virtual desktops and test/dev.

The virtual desktop (just in time desktops) use case is pretty obvious… You create that parent VM, spin it up and it gets quiesced and you can start forking that parent when needed. This will almost be instant, very efficient and also reduce the required resource capacity for VDI environments.

With test/dev scenarios you can imagine that when testing software you don’t want to wait for lengthy cloning processes to finish. Forking a VM will allow you to rapidly test what has been developed , within seconds you have a duplicate environment which you can use / abuse any way you like and destroy it when done. As the disk footprint is small, create/destroy will have a minimal impact on your existing infrastructure both from a resource and “stress” point of view. It basically means that your testing will take less time “end-to-end”.

Can’t wait for it to be available and to start testing it, especially when combined with products like CloudVolumes and Virtual SAN this feature has a lot of potential.

** UPDATE: Various people asked questions around what would happen with VMFork now TPS is disabled by default in upcoming versions of vSphere. I spoke with the lead engineer on this topic and he assured me there is no impact on VMFork. The disabling of TPS will be overruled per VMFork group. So the parent and childs belonging to the same group will be able to leverage TPS and share pages. **

RE: The VCDX candidates advantage over the panellists

· ·

I was reading Josh Odger’s post on the VCDX Defense. Josh’s article can be summarised with the following part:

As a result, the candidate should be an expert in the design being presented and answering questions from the panel about the design should not be intimidating.

Having gone through the process myself, knowing many of the VCDX’s and having been on countless of panels I completely disagree with Josh. Sure, you do need to know your design inside/out… but, it is not about “who’s having an advantage”, the panel member is not there to fail or pass the candidate… they are there to assess your skills as an architect!

If you look at the defense day there are three parts:

  1. Defend your design
  2. Design scenario
  3. Troubleshooting scenario

For the design and troubleshooting scenario you get a random exercise, so you have no prior knowledge of what will be asked. When it comes to defending your design of course you will know your design (hopefully) better then anyone else. However, the questions you get will not necessarily be about the specifics or details of your design. The VCDX panel is there to assess your skills as an architect and not your “fact cramming skills”. A good panel will ask a lot of hypothetical questions like:

  • Your design uses NFS based storage, how would FC connected storage have changed your design?
  • Your design is based on capacity requirements for 80 virtual machine, what would  you have done differently when the requirement would be 8000 virtual machines?
  • Your design …

So when you do mock exams, prepare for these types of hypothetical questions. That is when you really start to understand the impact decisions can have, and when during your defense you get one of these questions and you do not know the answer make sure you guide the panel through your thought process. That is what differentiates someone who can learn facts (VCP exam) and someone who can digest them, understand them and apply them in different scenarios (VCDX exam).

As I stated, it may sound like that you knowing your design inside out means having a big advantage over the panel members but it probably isn’t… that is not what they are testing you on! Your ability to assess and adapt are put through the wringer, your skills as an architect are tested thoroughly and that is where you will need to do well.

Good luck!

VMware patches for #shellshock

· ·

Last night a whole bunch of patches for the shellshock security issue were released. Although I am hoping that all of you have your datacenter secured for outside threads and inside threads by isolating networks, firewalls etc… It would be wise to install these patches ASAP. Majority of linux based VMware appliances were impacted, but luckily patching them is not a huge thing. Below you can find a list of the patches and links to the downloads for your convenience.

Note that the downloads are in the middle of the list, so you need to scroll down before you see them. There are also patches for products like the VMware VSA, vSphere Replication, VC Ops etc. Make sure to download those as well!

Changes – Joining Office of CTO

· ·

Almost 2 years ago I joined Integration Engineering (R&D) within VMware. As part of that role within Integration Engineering I was very fortunate to work on a very exciting project called “MARVIN”, as most of you know MARVIN became EVO:RAIL, which is what was my primary focus for the last 18 months or so. EVO:RAIL evolved in to a team after a successful prototype and came “out of stealth” at VMworld when it is was announced by Pat Gelsinger. Very exciting project, great opportunity and an experience I would not have wanted to miss out on. Truly unique to be one of the three founding members and see it grow from a couple of sketches and ideas to a solution. I want to thank Mornay for providing me the opportunity to be part of the MARVIN rollercoaster ride, and the EVO:RAIL team for the ride / experience / discussions etc!

Over the last months I have been thinking about where I wanted to go next and today I am humbled and proud to announce that I am joining VMware’s Office of CTO (OCTO as they refer to it within VMware) as a Chief Technologist. I’ve been with VMware little over 6 years, and started out as a Senior Consultant within PSO… I never imagined, not even in my wildest dreams, that one day I would have the opportunity to join a team like this. Very honoured, and looking forward to what is ahead. I am sure I will be placed in many uncomfortable situations, but I know from experience that that is needed in order to grow. I don’t expect much to change on my blog, I will keep writing about products / features / vendors / solutions I am passionate about. That definitely was Virtual SAN in 2014, and could be Virtual Volumes or NSX in 2015… who knows!

Go OCTO!