• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • Unexplored Territory Podcast
  • HA Deepdive
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

5.0

5 is the magic number

Duncan Epping · Jul 12, 2011 ·

Yes, here it is… the moment we’ve all been waiting for… vSphere 5.0. Finally announced today, and on top of that also new releases of vCD (1.5), SRM (5.0), vShield (5.0) and of course a new product called the vSphere Storage Appliance. Over the last months I have been working hard on collateral for this launch and soon it should be available. For me personally vSphere 5.0 is most definitely the launch that had the most impact ever. Not only did I work on some of the material that will be released, I also prepared roughly 20 blog articles which will be released over the upcoming weeks and lets not forget the vSphere 5.0 Clustering Tech Deepdive that we’ve released today. Crazy times indeed, but that’s not the topic of this article. Lets talk about vSphere 5.0 for a bit and why I am so excited about this release in particular. I could do a copy of past of the releasenotes, which I am certain many will do, but where’s the fun in that? Instead I am going to list some of the changes in vSphere 5.0 and mention why I feel these are important. I won’t go into much detail yet as that will more than likely happen in one of the upcoming articles.

I have picked 5 topics which I feel deserved to be called out and which are enormous!

  1. Storage DRS
  2. vSphere HA aka FDM
  3. Profile-Driven Storage
  4. vSphere Storage APIs
  5. Stateless

Storage DRS

This is near and dear to my heart. I have been involved with SDRS very closely over the last 8 months and provided a lot of feedback around the UI, some which has made it into this release. So what is Storage DRS and why do you need it? Storage DRS does for storage what DRS does for compute resources. Storage DRS allows you to aggregate storage resources (datastores) into a single object called a Datastore Cluster. This datastore cluster is the object you will need to manage from now on. SDRS will balance resources (aka virtual machines) within a datastore cluster. In other words, SDRS allows you to specify a latency and space utilization threshold and based on that it will make balancing recommendations. It can do this fully automated and that probably sounds very compelling to most of you, and it is. However I feel that the true strength of SDRS is Initial Placement. Some of you might recognize this, and some of you unfortunately won’t, but when I deployed virtual machines I would check all VMFS volumes first and identify those with the most available disk space and than check the average latency to make sure I wasn’t creating hot spots. That was a fairly cumbersome task to be honest and that is exactly what SDRS also solves. On top of that it offers things like “Datastore Maintenance Mode” and (Anti) Affinity Rules. Definitely a feature worth evaluating in my opinion and a feature which would justify an upgrade to Enterprise+ due to the reduction in operational effort and the possible problems/bottlenecks it can detect and help preventing.

vSphere HA aka FDM

This is one of those features that many take for granted. HA just works doesn’t it…. But all of us have seen some of the constraints that came with it like the max of 5 primary nodes. All of that has been solved with vSphere 5.0. vSphere HA has been rewritten from the ground up. Yes I do mean from the ground up, AAM is dead and FDM is introduced. FDM stands for Fault Domain Manager and is the name of the new agent. From a UI perspective not a lot has changed however, except for a couple things that appear to be minor but are major in my opinion like: Datastore Heartbeating and Admission Control Policies. Datastore Heartbeating allows HA to make a distinction between a host that is isolated and between a host that has failed. You might wonder why that is important, well in the past HA would try restarting VMs regardless of the state of the host and this causes some overhead and even problems in the past. That has now been solved. On top of that the Admission Control Policies were improved. The percentage based admission control policy allows you to specify a separate percentage for both CPU and memory. There’s much more under the covers that has changed though, no more primary/secondary node concept as stated but a master/slave concept with an automated election process. Anyway, too much details for now, will come back to that later.

Profile-Driven Storage

Profile-Driven Storage is the future. In an ideal world admins are maintaining massive spreadsheets detailing storage characteristics and VM storage requirements. This spreadsheet should be used during the provisioning and during the migration of VMs, but we all know that in many cases placement is either at random or “best of knowledge”. In both cases, and even when a spreadsheet/database is used, this leads to human error or a serious operational overhead. With vSphere 5.0 that is not necessary anymore as Profile-Driven Storage helps preventing these errors. Profile-Driven Storage, in the UI referred to as VM Storage Profile, allows you to create a profile with specific characteristics. This profile can simply be linked to a VM/VMDK and will allow you to do compliancy checks. Simple solution, but very effective and especially when combined with one of the new vSphere Storage APIs…

vSphere Storage APIs

Most of you have seen VAAI into action by now. We all know how it can help reducing the time to make a clone, deploy a template, create an eager zero thick disk and of course offload the locking mechanism. With 5.0 this has been expanded and enhanced. First and foremost, in my opinion, all primitives (features) are now T10 compliant. This means that every single storage vendor out there that adheres to the T10 standards can benefit from VAAI without the need to write their own plugin. On top of that a Thin Provisioning primitive is introduced which does two things: reclaim dead space and provide out-of-space info. Reclaim dead space useful in environments where thin provisioned LUNs are used and VMs are often deleted. This will allow the array to reclaim the blocks associated with the LUN when they are no longer in use. Out-of-space info, well I guess you know what that does… provide details around utilization on the backend (array) to the frontend (vCenter/ESXi) and allows for alarms etc. On top of that a brand new feature is introduced called vSphere Storage API for Storage Awareness aka VASA. VASA surfaces storage characteristics to vCenter. Basically it enables you to retrieve storage details from the “storage provider” and use this for Profile-Driven Storage, SDRS etc. Think about details like RAID level used, thin / thick, deduped, replication etc.

Stateless

Last but not least, support for Stateless. This allows you to run diskless setups and boot ESXi into memory over the network. vSphere 5.0 provides the Auto Deployment tool which allows you to manage stateless ESXi environments. The cool thing in my opinion about stateless ESXi, keep in mind that “state” is stored by Auto Deployment and vCenter, is that is makes the update process extremely easy. Instead of patching dozens of hosts you patch the main image and just reboot your host whenever you please. Is that agile or what?

Wrapping up

There’s much more of course to vSphere 5.0 than I have touched on today. However, I suspect that the whole blogosphere will be swamped with blog articles and as such there is no point in calling out every single cool detail as they will drown in a large amount of info floating around. These 5 are my personal favorites mainly because they reduce the amount of operational effort required to run a virtualized infrastructure. I cannot wait for the product to be available and hear all your responses!

 

** Disclaimer: This article contains references to the words master and/or slave. I recognize these as exclusionary words. The words are used in this article for consistency because it’s currently the words that appear in the software, in the UI, and in the log files. When the software is updated to remove the words, this article will be updated to be in alignment. **

Live Blog: Raising The Bar, Part V

Duncan Epping · Jul 12, 2011 ·

I am live at the Launch event in San Francisco with many other bloggers, journalists and analysts. It is the 12th of July, almost 09:00 PDT and Paul Maritz is about come up on stage to talk about the Cloud Infrastructure launch. This article will be update live during the event as we go.

Paul Maritz is taking the stage… Taking the next step in towards the more automated world.

We need to make infrastructure become something that people can depend on and focus on what is important to their business. Navigating your way forward offering a more dynamic infrastructure that will support your existing applications. Using a more flexible infrastructure, allowing people to take resources and aggregate to larger pools reducing operational costs by automating the use of these resources. More and more use of social media and use of mobile devices to connect anytime anywhere and most importantly securely.

Today we will be talking about  a more efficient infrastructure with exists of three stages IT Production, Business Production and IT as a Service. In 2009, the VI 3 era, 30% of the workloads were virtualized…. in 2010 with vSphere 4 we reached 40% and it is expected that in 2011 we will hit 50% virtualized with the majority on vSphere.

Accelerating and Amplifying business agility with vSphere 5! Not only vSphere 5 but the worlds first Cloud Infrastructure suite! In addition to vSphere 5 today we announce vSphere Site Recovery Manager 5 (Business Continuity), vCloud Director 1.5 (Policy, Reporting, Self-Service), vCenter Operations 1.0.1 (Monitoring and Management), vShield 5 (Security and Edge functionality).

VMware vCloud = Hybrid. Your private cloud experience needs to be similar to public cloud experience. VMware allows this through the vCloud offering and vCloud Service Providers. Trusted vCloud partners like Colt, Bluelock, Singtel, Verizon, NYSE Euronext, Softbank and CSC are some of the enablers for this.

Steve Herrod up on stage… I expect it is about to get more technical

Why do these new products matter and how do they fit together. Accelerating the adoption by increasing scalability. ESX 1.0 capable of 5000 IOps, ESX 2.0 ~ 7000 IOps, VI 3 100.000 IOps, vSphere 4 300.000 IOps and vSphere 5.0 1.000.000 IOps. Besides performance availability is key. Both HA and FT have been enhanced and of course SRM 5.0 has been released. Added to SRM 5.0 is vSphere Replication. vSphere Replication allows you to use the network to replicate between sites and different arrays. It will allow you to replicate more workloads with a lower costs. SRM is about datacenter mobility, not only for an outage but also pro-actively moving datacenters after an acquisition.

What does cloud computing really mean? Being able to order what you need and what without knowing what happens behind the scenes. IT will behind the scenes validate if they meet the consumers requirements. vCloud Director is all about Simple Self-Service. Deploy virtual machines but more importantly create new vApps and offer these in your own “app store”. The IT Cloud of the producer is all about offering agility. Virtualization enables automation in a way unheard in a physical environment.

Typically multiple tiers are offered within a cloud environment. The VMware Cloud Infrastructure enable you to do so. Intelligent Policy based Management is key with vCloud Director 1.5. Linked Clones is a very important feature to provision virtual machines “aggressively” within the system. It allows for fast provisiong and save up to 60% of storage.

Profile-Driven Storage and Storage DRS are part of vSphere 5.0. It enables you to map different arrays in to logical entities by a concept called a “datastore cluster” and link these to a profile. Virtual machines will be tagged with a profile and this allows you to validate compliancy. Storage DRS does for storage what DRS does for compute resources. Storage and Network IO Control ensures each virtual machine receives what it is entitled to.

For the SMB market a brand new shared storage appliance is introduced today: vSphere Storage Appliance 1.0. It takes vanilla servers and use local drives and present it as shared storage. It will bring agility and availability through shared storage to the SMB.

Auto-Deploy, PXE booting your ESXi hypervisor in to memory! It allows to spin-up more hosts within minutes instead of hours. Adding capacity has never been this simple?

vSphere 5 offers comprehensive security and isolation capabilities through vShield 5.0. vShield App 5 allows you to select regulations to protect sensitive data. It also enables you to get additional auditing in place.

The Cloud Infrastructure represents more than a million engineering hours, more than 100 additional capabilities, more than two million QA hours, more than 2000 partner certifications to enable this.

Rick Jackson up next discussing licensing.

Industry has traditionally licensed on physical constraints. It makes it difficult to create a cloud environment. Customers need to be able to upgrade to new hardware without having physical boundaries. No more “Cores per Proc” limits, no more “Physical RAM per host license”… vSphere introducing vRAM entitlement. Virtual RAM is the amount of virtual memory configured for a powered on virtual machine. vSphere 5 used pooled vRAM across the entire environment.

Packaging has been simplified and moving from 6 down to 5 packages. vSphere Advanced has been eliminated, all customers currently using Advanced are entitled to vSphere Enterprise.

Join us at VMworld for more details around the new product releases. 10AM virtual show, be there for more technical in-depth details!

Hot of the press: vSphere 5.0 Clustering Technical Deepdive

Duncan Epping · Jul 12, 2011 ·

** Update: Available now: paperback full |paperback black & white **

After months of hard work the moment is finally there, the release of our new book: vSphere 5.0 Clustering Technical Deepdive! When we started working, or better said, planning an update of the book we never realized the amount of work required. Be aware that this is not a minor update. This book covers HA (full rewrite as HA has been rewritten for 5.0), DRS (mostly rewritten to focus on resource management) and Storage DRS (new!). Besides these three major pillars we also decided to add what we call supporting deepdives. The supporting deepdives added are: vMotion, Storage vMotion, Storage I/O Control and EVC. This resulted in roughly 50% more content (totaling 348 pages) than the previous book, also worth noting that every single diagram has been recreated and are they cool or what?

Before I will give you the full details I want to thanks a couple of people who have helped us tremendously and without whom this publication would not have been possible. First of all I would like to thank my co-author Frank “Mr Visio” Denneman for all his hard work. Frank and I would also like to thank our VMware management team for supporting us on this project. Doug “VEEAM” Hazelman thanks for writing the foreword! A special thanks goes out to our technical reviewers and editors: Doug Baer, Keith Farkas and Elisha Ziskind (HA Engineering), Anne Holler, Irfan Ahmad and Rajesekar Shanmugam (DRS and SDRS Engineering), Puneet Zaroo (VMkernel scheduling), Ali Mashtizadeh and Gabriel Tarasuk-Levin (vMotion and Storage vMotion Engineering), Doug Fawley and Divya Ranganathan (EVC Engineering). Thanks for keeping us honest and contributing to this book.

As promised in the multiple discussions we had around our 4.1 HA/DRS book we wanted to make sure to offer multiple options straight away. While Frank finalized the printed copy I worked on formatting the ebook. Besides the black&white printed version we are also offering a full color version of the book and a Kindle version. The black&white sells for $ 29.95, the full color for $ 49.95 and the Kindle for an ultra cheap price: $ 9.95. Needless to say that we recommend the Kindle version. It is cheap, full color and portable or should we say virtual… who doesn’t love virtual? On a sidenote, we weren’t planning on doing a black and white release but due to the extremely high production costs of the full color print we decided to offer it as an extra service. Before I give the full description here are the direct links to where you can buy the book. (Please note that Amazon hasn’t listed our book yet, seems like an indexing issue, should be resolved soon hopefully For those who cannot wait to order the printed copy check-out Createspace or Comcol.

Amazon:
eBook (Kindle) – $ 9.99
(price might vary based on location as amazon charges extra for delivery)
Black & White Paper – $ 29.95
Full Color Paper – $ 49.95

Createspace:
Black & White Paper – 29.95
Full Color Paper – 49.95

For the EMEA folks comcol.nl offered to distribute it again, paper black & white can be found here, and full color here.

VMware vSphere 5.0 Clustering Technical Deepdive zooms in on three key components of every VMware based infrastructure and is by no means a “how to” guide. It covers the basic steps needed to create a vSphere HA and DRS cluster and to implement Storage DRS. Even more important, it explains the concepts and mechanisms behind HA, DRS and Storage DRS which will enable you to make well educated decisions. This book will take you in to the trenches of HA, DRS and Storage DRS and will give you the tools to understand and implement e.g. HA admission control policies, DRS resource pools, Datastore Clusters and resource allocation settings. On top of that each section contains basic design principles that can be used for designing, implementing or improving VMware infrastructures and fundamental supporting features like vMotion, Storage I/O Control and much more are described in detail for the very first time.

This book is also the ultimate guide to be prepared for any HA, DRS or Storage DRS related question or case study that might be presented during VMware VCDX, VCP and or VCAP exams.

Coverage includes:
– HA node types
– HA isolation detection and response
– HA admission control
– VM Monitoring
– HA and DRS integration
– DRS imbalance algorithm
– Resource Pools
– Impact of reservations and limits
– CPU Resource Scheduling
– Memory Scheduler
– DPM
– Datastore Clusters
– Storage DRS algorithm
– Influencing SDRS recommendations

Be prepared to dive deep!

Pick it up, leave a comment and of course feel free to make those great mugshots again and ping them over via Facebook or our Twitter accounts! For those looking to buy in bulk (> 20) contact clusteringdeepdive@gmail.com.

  • « Go to Previous Page
  • Page 1
  • Interim pages omitted …
  • Page 31
  • Page 32
  • Page 33

Primary Sidebar

About the Author

Duncan Epping is a Chief Technologist and Distinguished Engineering Architect at Broadcom. Besides writing on Yellow-Bricks, Duncan is the co-author of the vSAN Deep Dive and the vSphere Clustering Deep Dive book series. Duncan is also the host of the Unexplored Territory Podcast.

Follow Us

  • X
  • Spotify
  • RSS Feed
  • LinkedIn

Recommended Book(s)

Advertisements




Copyright Yellow-Bricks.com © 2025 · Log in