Startup Into: Platform9

Yesterday a startup came out of stealth which was founded by a couple of former VMware veterans. I happen to know the majority of them, and have had the pleasure to have worked with them on various things in the past. For instance Platform9‘s three of the four co-founders were all heavily involved in vCloud Director, and the fourth co-founder was VMware employee number 27… but that is not where it stops… there is much more talent on-board. But that is not what this blog is about, this blog is about Platform9, the new company that they have formed and the product they are building.

** note, I did not test the product… it is impossible to provide an analysis of what works / does not work and how they play in this space or compete with others, this is based on a conversation and a demo **

Platform9 is as they say themselves:

… the easiest way for enterprises to implement a private cloud, with intelligent, self-service provisioning of workloads onto their computing infrastructure.

  • 100% Cloud Managed: Platform9′s cloud-based model means that there is no complex management software to setup, monitor and upgrade, thus simplifying the operational experience.
  • Single Pane of Glass: Platform9 offers unified management across diverse environments – Docker, KVM and VMware vSphere – across datacenters and geographies.
  • Based on OpenStack: Platform9 customers get the best of OpenStack with 100% API compatibility.

When I met them last month, they gave a demo and showed me what they had implemented so far: Management of KVM based hypervisors using a very easy to use and slick using web-based user interface. Where the whole management solution was running in “the cloud”.

Creating a new “instance” was literally a few clicks, snapshotting / cloning it… same thing, just a couple of clicks. Now what stood out to me during the demo was the use of the word “instance” instead of “virtual machine”. So I asked them why not “virtual machine”, considering they are all VMware veterans that made more sense to me. The explanation was simple: we want to manage multiple layers. We want to manage KVM VMs, vSphere VMs but for instance also Docker containers. That is why we used a different term than we would normally use… and yes that did make sense to me. I also wondered if they would be able to mix different environments and type of instance in their UI, and the answer was yes. Docker containers, KVM VMs, vSphere VMs (etc) will also be seen in the same single pane of glass. I really like the fact that Platform9 did not limit themselves to just vSphere, or just VMs but rather is focusing on the needs of developers and providing what they require.

Similar to CloudPhysics, Platform9 is taking SaaS approach. The major benefit of course being the agility at which new features/functionality can be introduced to the outside world, or tested against a small subset of customers. Same of course applies to bug fixes / updates. No need to do that yourself, Platform9 will take care of that for you. Promising indeed.

Now there is a lot of competition in this space, as also emphasized by  Ben Kepes in his post on Platform9… But to be honest, if I look at one of the examples listed like ServiceMesh, they seem to cater for a completely different market. Platform9 is all about simplicity and managing instances, not so much about creating complex recipes etc. I agree though that there are a lot of vendors playing in this space (and as such competition), or somewhat related space, but that makes it even more interesting to see how Platform9 evolves in my opinion.

For more info, a demo, or a trial:

Platform9 will showcase its product in its booth #324 at VMware’s VMworld Conference, taking place the week of August 25th in San Francisco. The product is currently in beta with general availability planned for later this year. Platform9 currently supports KVM with upcoming support for Docker and VMware vSphere. To register for a free trial, go to:

vSphere 5.5 and disk limits (mClock scheduler caveat)

I mentioned the new disk IO scheduler in vSphere 5.5 yesterday. When discussing this new disk IO scheduler one thing that was brought to my attention is a caveat around disk limits. Lets get started by saying that disk limits are a function of the host local disk scheduler and not, I repeat, not Storage IO Control. This is an often made mistake by many.

Now, when setting a limit on a virtual disk you define a limit in IOPS. The IOPS specified is the maximum number of IOPS the virtual machine can drive. The caveat is is as follows: IOPS takes the IO size in to account. (It does this as a 64KB IO has a different cost than a 4KB IO.) The calculation is in multiples of 32KB. Note that if you do a 4KB IO it is counted as one IO, however if you do a 64KB IO it is counted as two IOs. Any IO larger than 32KB will be 2 IOs at a minimum as it is rounded up.  In other words, a 40KB IO would be 2 IOs and not 1.25 IOs. This also implies that there could be an unexpected result when you have an application doing relatively large blocksize IOs. If you set a limit of 100 IOPS but your app is doing 64KB IOs than you will see your VM being limited to 50 IOPS as each 64KB IO will count as 2 IOs instead of 1. So the formula here is: ceil(IO Size / 32).

I think that is useful to know when you are limiting your virtual machines. Especially cause this is a change in behaviour compared to vSphere 5.1.

New disk IO scheduler used in vSphere 5.5

When 5.1 was released I noticed the mention of “mClock” in the advanced settings of a vSphere host. I tried enabling it but failed miserably. A couple of weeks back I noticed the same advanced setting again, but this time also noticed it was enabled. So what is this mClock thingie? Well mClock is the new disk IO scheduler used in vSphere 5.5. There isn’t much detail on mClock by itself other than an academic paper by Ajay Gulati.

disk io scheduler

The paper describes in-depth why mClock was designed / developed, it primarily was to provide a better IO scheduling mechanism that allows for limits, shares and yes also reservations. The paper also describes some interesting details around how different IO sizes and latency is taken in to account. I recommend anyone who likes reading brain hurting material to take a look at it. I am also digging internally for some more human readable material, If I find out more I will let you guys know!

Essential Virtual SAN book available as of today! (ebook first)

Yes, the day has finally come… Our pet project, the Essential Virtual SAN book is finally out! Cormac and I decided to take the “e-book first” route which enables us to have it out weeks before the printed copy. Before doing the thank you’s and provide you with some details on what the book is about, I want to thank my co-author Cormac! It was a great pleasure working with you on this project Cormac, thanks for asking me to be part of this exciting book!

We want to thank our technical editors Paudie O’Riordan and Christos Karamanolis, whom spent countless of hours reading and editing our raw materials. We would like to thank the VMware Virtual SAN engineering team for the countless of hours discussing the ins and outs of Virtual SAN. Especially Christian Dickmann and (again) Christos Karamanolis, it would not have been possible without your help! We also want to acknowledge William Lam, Wade Holmes, Rawlinson Rivera, Simon Todd, Alan Renouf, and Jad El-Zein for their help and contributions to the book. Last but not least we want to thank the Pearson team for their flexibility and agility and getting things done, and our management (Phil Weiss, Adam Zimman, and Mornay van der Walt) for supporting us on this journey.!

Cormac and I are also very pleased to say that we have two awesome forewords by no one less than VMware CTO Ben Fathi and SVP of Storage and Availability at VMware Charles Fan! Thanks for taking the time out of your busy schedule, we very much appreciate it.

What does the book cover?

Understand and implement VMware Virtual SAN: the heart of tomorrow’s Software-Defined Datacenter (SDDC)

VMware’s breakthrough Software-Defined Datacenter (SDDC) initiative can help you virtualize your entire datacenter: compute, storage, networks, and associated services. Central to SDDC is VMware Virtual SAN (VSAN): a fully distributed storage architecture seamlessly integrated into the hypervisor and capable of scaling to meet any enterprise storage requirement.

Now, the leaders of VMware’s wildly popular Virtual SAN previews have written the first authoritative guide to this pivotal technology. You’ll learn what Virtual SAN is, exactly what it offers, how to implement it, and how to maximize its value.

Writing for administrators, consultants, and architects, Cormac Hogan and Duncan Epping show how Virtual SAN implements both object-based storage and a policy platform that simplifies VM storage placement. You’ll learn how Virtual SAN and vSphere work together to dramatically improve resiliency, scale-out storage functionality, and control over QoS.

Both an up-to-the-minute reference and hands-on tutorial, Essential Virtual SAN uses realistic examples to demonstrate Virtual SAN’s most powerful capabilities. You’ll learn how to plan, architect, and deploy Virtual SAN successfully, avoid gotchas, and troubleshoot problems once you’re up and running.

Coverage includes

  • Understanding the key goals and concepts of Software-Defined Storage and Virtual SAN technology
  • Meeting physical and virtual requirements for safe Virtual SAN implementation
  • Installing and configuring Virtual SAN for your unique environment
  • Using Storage Policy Based Management to control availability, performance, and reliability
  • Simplifying deployment with VM Storage Policies
  • Discovering key Virtual SAN architectural details: caching I/O, VASA, witnesses, pass-through RAID, and more
  • Ensuring efficient day-to-day Virtual SAN management and maintenance
  • Interoperating with other VMware features and products
  • Designing and sizing Virtual SAN clusters
  • Troubleshooting, monitoring, and performance optimization

ISBN-10: 013385499X
ISBN-13: 978-0133854992

You can buy it via for Kindle, and it will also be available shortly via for any other ebook format!

das.maskCleanShutdownEnabled is set to true by default

I had a couple of questions on the topic of das.maskCleanShutdownEnabled today. For those who have not read the other articles I wrote about this topic, this is in short what it does and why it was introduced and how I explained it in an email today:

When a virtual machine is powered off (or shut down) by a user a property is set to true named runtime.cleanPowerOff”. To vSphere HA this indicates that the virtual machine was powered off by a user and as such when a host fails it knows that for this virtual machine it doesn’t need to take action. By default this property is set to true. If for whatever reason the virtual machine is killed by ESXi than this property is set to false.

vSphere HA provides the ability to respond to a storage failure (PDL). When a PDL occurs it can kill a virtual machine and then restart the virtual machine. However, runtime.cleanPowerOff” default is “true” and vSphere HA cannot access the datastore (PDL remember) to change the property! So this means if the VM is killed after the PDL, then it won’t be restarted as HA assumes it was cleanly powered off.

This is where das.maskCleanShutdownEnabled comes in to play. By setting this to “true”, vSphere HA assumes that VM is not cleanly powered off. Only when you cleanly power it off the property is set. In other words, In a PDL situation it will now restart the VM even though the datastore was unavailable when the VM was killed!

Back to the original question, what is das.maskCleanShutdownEnabled set to in 5.1 and later? Do you need to set it manually? No you do not, by default it is set to true! So when you configure a cluster, be aware of this… Especially in a stretched cluster environment where a PDL scenario is not unlikely.

** do not forget to also set terminateVMonPDL described in this blog post if you want VMs to be automatically killed when a PDL occurs! **