• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

analytics

CloudPhysics card builder, how awesome is that?

Duncan Epping · Jul 2, 2013 ·

A while ago Irfan Ahmad (CloudPhysics CTO), Frank Denneman and I were discussing various ideas around the CloudPhysics platform… One of the ideas that Ifran and team pitched was this notion of a card builder. Both Frank and I are advisors to CloudPhysics and immidiately jumped up and said “YES PLEASE, when can we have it?” Over the last couple of weeks you have probably seen various blog posts pop up about the card builder that CloudPhysics created and I can honestly say that it has exceeded my expectations. (Suggested reads: Willam’s blog post, Anthony Spiteri’s post) So what is so special about this card designer? I think this paragraph from William’s blog post describes it best:

The vSphere platform provides a very powerful and rich set of APIs (Application Programming Interface) that can be consumed by both vSphere administrators as well as developers. However, there is a high learning curve when using the API and it takes quite a bit of time to learn and of course your manager is expecting the report to be done in the next 5 minutes. Even with abstraction tools such as PowerCLI, quickly building a robust, scalable and performant script is not always a trivial task, not to mention the maintenance and updates to the script because your manager wants to continually add more things to the report.

Not everyone is an API guru like William or a scripting god like Alan Renouf or Luc Dekens. Sure, these guys will knock out an awesome looking report in a matter of minutes, maybe 10 – 15 minutes depending on what kind of metrics they need and how complex the report will be. For normal people, like myself, who aren’t scripting gods this typically takes a lot longer. Personally I am happy if I can produce something within an hour, but when it gets more complex you are probably talking about way more than that, potentially a full day. The CloudPhysics card builder was designed to lower the barrier to create meaningful reports!

How simple is it? I would say, that if I can figure it out in seconds it is dead simple:

  1. Click “Card Builder”
    CloudPhysics Card Builder
  2. Click “Create card”
    CloudPhysics Card Builder
  3. Select the “Property”
    CloudPhysics Card Builder
  4. I selected “Datastore:Name” and “Datastore:Attached Hosts” and below the results
    CloudPhysics Card Builder

That is it, really easy right? In just a couple of clicks I can see which hosts are connected to which datastores. Yes of course this was a simple example, but the nice thing is that you can make it as complex as you want or need. Currently this is in a limited Beta, but soon (I mean really soon!!) this will be exposed to the rest of the world. If you want to know more, just check the webinar recording by Irfan link can be found on the CPhy website!

Only thing I wonder is… why on earth did no one come up with this concept before for the virtualization space? Creating reports and should always be dead simple if you ask me, and now with CloudPhysics Card Builder it finally is.

The State of vSphere Clustering by @virtualirfan

Duncan Epping · Oct 23, 2012 ·

The state of vSphere clustering
By Irfan Ahmad

Some of my colleagues at CloudPhysics and I spent years at VMware and were lucky to have participated in one of the most rapid transformations in enterprise technology history. A big part of that is VMware’s suite of clustering features. I worked alongside Carl Waldspurger in the resource management team at VMware that brought to the world the ESX VMkernel CPU and memory schedulers, DRS, DPM, Storage I/O Control and Storage DRS among other features. As a result, I am especially interested in analyzing and improving how IT organizations use clustering.

Over a series of blog posts, I’ll try to provide a snapshot of how IT teams are operationalizing vSphere.  One of my co-founders, Xiaojun Liu and I performed some initial analysis on the broad community dataset that is continually expanding as more virtualization engineers securely connect us to their systems.

First, we segmented our analysis based on customer size. The idea was to isolate the effect of various deployment sizes including test labs, SMBs, commercial and large enterprise, etc. Our segmentation was in terms of total VMs in customer deployments and divided up as: 1-50 VMs, 51-200, 201-500, 501-upwards. Please let us know if you believe an alternative segmentation would warrant better analysis.

Initially we compared various ESX versions deployed in the field. We found ESXi 5.0 already captured the majority of installations in large deployments. However, 4.0 and 3.5 versions continue to be deployed in the field in small numbers. Version 4.1, on the other hand, continues to be more broadly deployed. If you are still using 4.1, 4.0, and 3.5, we recommend upgrading to 5.0 which provides greatly improved HA clustering, amongst many other benefits. This data shows the 5.0 version has been broadly adopted by our peers and is user-verified production ready.

Next, we looked at cluster sizes. A key question for VMware product managers was often, “How many hosts are there in a typical cluster?” This was a topic of considerable debate, and it is critically important to know when prioritizing features. For example, how much emphasis should go into scalability work for DRS.

For the first time, CloudPhysics is able to leverage real customer data to provide answers. The highest frequency cluster size is two hosts per cluster for customers with greater than 500 VMs. Refer to the histogram. This result is surprisingly low and we do not yet know all the contributing reasons, though we can speculate on some of the causes. These may be a combination of small trainiång clusters, dedicated clusters for some critical applications, Oracle clustering license restrictions, or perhaps a forgotten pair of older servers. Please tell us why you may have been keeping your clusters small.

Despite the high frequency of two-host clusters, we see opportunities for virtualization architects to increase their resource pooling. By pooling together hosts into larger clusters, DRS can do a much better job at placement and providing resource management. That means real dollars in savings. It also allows for more efficient HA policy management since the absorption of spare capacity needed for infrequent host failures is now spread out over a larger set of hosts. Additionally, having fewer clusters makes for fewer management objects to configure, keep in sync with changing policies, etc. This reduces management complexity and makes for a safer and more optimized environment.

Several caveats arise with regard to the above findings. First is potential sample bias. For instance, it might be the case that companies using CloudPhysics are more likely to be early adopters and that early adopters might be more inclined to upgrade to ESX 5.0 faster. Another possible issue is imbalanced dataset composition. It might be that admins are setting up small training or beta labs, official test & development, and production environments mixed in the same environment thus skewing the findings.

CloudPhysics is the first to provide a method of impartially determining answers based on real customer data, in order to dampen the controversy.

Xiaojun and I will continue to report back on these topics as the data evolves. In the meantime, the CloudPhysics site is growing with new cards being added weekly. Each card solves daily problems that virtualization engineers have described to us in our Community Cards section. I hope you will take the time to send us your feedback on the CloudPhysics site.

CloudPhysics #VMworld challenge, win a Mac Pro/Air or Google Nexus

Duncan Epping · Aug 18, 2012 ·

CloudPhysics just launched their website and with it a beta version of their product. As a great incentive to get people started with their product they came up with a contest where you can earn points by describing your virtualization problems. You can win some nice prices (retina Mac Pro, Google Nexus 7, Mac Air), so make sure to get started soon.

Your score is a calculation of the number of Cards you propose, plus a sum of the amount of activity you spend in the CloudPhysics portal. To achieve the best possible score, be sure to install a CloudPhysics Observer vApp, propose several new cards, vote on cards, and try out all the features of the portal. The greater the amount of activity and engagement, the higher your score can grow.

If you want to get started, go over to app.cloudphysics.com/login and login. Make sure to download the appliance! Now when you have downloaded it and got it up and running you should see data coming in soon. But where you really start collecting points is when you  start suggesting Cards. Look at the Card below, this is where you can start making suggestions, just hover over the lower right of the card to get in to the system…

Each of your suggestions will result in points. On top of that you also get points awarded for using their appliance  and commenting on suggestions from other users. A simple, but very cool challenge. The cool thing about this challenge is that these suggestions could  make it in to the product. In other words, if there is a problem you have faced many times and would like to prevent others from hitting the same thing… Report it and challenge the CloudPhysics people to make it part of their offering!

Current top-3 (after the first day):

  1. Jake Robinson
  2. William Lam
  3. Alan Renouf

Surely you can beat these 3 guys and collect your price at #VMworld. Go for it 🙂

CloudPhysics, not so much stealth mode start-up anymore…

Duncan Epping · Jul 19, 2012 ·

<disclaimer: I am a technical advisor for CloudPhysics>

cloud physics logoToday at the New England VMUG CloudPhysics has their first official “public appearance”. Yes some of you have heard the name a couple of times before and some of you might even know who the brains are behind this new start-up… for those who don’t let me give a brief introduction.

CloudPhysics was recently founded by John Blumenthal and Irfan Ahmad. Some of you might recognize their names as they used to work at VMware, John was a Product Manager for storage and Irfan was the person who was responsible for awesome features like Storage DRS and Storage IO Control. Together with several other brilliant people, including no one less than Carl “TPS / DRS” Waldspurger acting as an advisor and consultant, they founded a new company.

So what is CloudPhysics about? CloudPhysics is about big data, about centralized data, about analytics, about modeling data. CloudPhysics is essentially about helping you! How? Well let me try to explain that without revealing too much.

We’ve all monitored and managed environments, some of you are responsible for 3 hosts and some might be responsible for 80 hosts in different sites and in different companies. We all face several challenges and in many cases these are similar… How do you find common themes? How do you validate best practices are applied on all levels in your environment? How do you validate if your practices are actually used by others, and do you benefit from them? How do you know if you sized correctly? How do I solve specific problems? Would I benefit from a different storage platform or SSD? All of these are questions or problems you probably face daily and that is where CloudPhysics aims to come in to play.

CloudPhysics will enable you to find common best practices and problems in your environment. CloudPhysics will provide you guidance, this could be custom but also generic through for instance a link to a VMware KB article. They will enable you to compare and explore performance results. Find patterns in your environment… See trends and provide you with meaningful statistics about your environment. Sounds amazing right and probably something you wouldn’t mind testing today… The CloudPhysics product will come as a virtual appliance. The data gathered will go up to the cloud and all of the analysis will happen outside of your environment, of course with various degrees of anonymity.

CloudPhysics is constructing an analytics platform for vSphere for the application of collective intelligence to individual, local vSphere environments and users.  At the same time the platform is intended to service the needs of consulting companies, customers and the blogging community by providing APIs to enable unique exploration and discovery within the dynamic, changing dataset CloudPhysics continuously generates. Access to this dataset enables them to transform qualitative discussions into quantitative views of vSphere design and operation. CloudPhysics is not seeking to build a community; rather, it exists to empower the engineer and architect in all of us, particularly the commentators and critics essential to the industry.

For those who can’t wait, sign up at www.cloudphysics.com now for announcements and news on the beta. I am excited about CloudPhysics and I hope you all are as well.

Primary Sidebar

About the author

Duncan Epping is a Chief Technologist in the Office of CTO of the Cloud Platform BU at VMware. He is a VCDX (# 007) and the author of the "vSAN Deep Dive" and the “vSphere Clustering Technical Deep Dive” series.

Upcoming Events

06-May-21 | VMUG Iceland- Roadshow
19-May-21 | VMUG Saudi – Roadshow
26-May-21 | VMUG Egypt – Roadshow
27-May-21 | VMUG Australia – Roadshow

Recommended reads

Sponsors

Want to support us? Buy an advert!

Advertisements

Copyright Yellow-Bricks.com © 2021 · Log in