Storage DRS interoperability

I was asked about this a couple of times over the last few days so I figured it might be an interesting topic. This is described in our book as well in the Datastore Cluster chapter but I decided to rewrite it and add some of it into a table to make it easier to digest. Lets start of with the table and explain why/where/what… Keep in mind that this is my opinion and not necessarily the best practice or recommendation of your storage vendor. When you implement Storage DRS make sure to validate this against their recommendations. I have marked the area where I feel caution needs to be taken with (*).

Capability Mode Space I/O Metric
Thin Provisioning Manual Yes (*) Yes
Deduplication Manual Yes (*) Yes
Replication Manual (*) Yes Yes
Auto-tiering Manual Yes No (*)

Yes you are reading that correctly, Storage DRS enabled with all of them and even with I/O metric enabled except for auto-tiering. Now although I said “Manual” for all of them I even believe that in some of these cases Fully Automated mode would be perfectly fine. Now as it will of course depend on the environment I would suggest to start out in Manual mode if any of these 4 storage capabilities are used to see what the impact is after applying a recommendation.

First of all “Manual Mode”… What is it? Manual Mode basically means that Storage DRS will make recommendations when the configured thresholds for latency or space utilization has been exceeded. It also will provide recommendations for placement during the provisioning process of a virtual machine or a virtual disk. In other words, when setting Storage DRS to manual you will still benefit from it as it will monitor your environment for you and based on that recommend where to place or migrate virtual disks to.

In the case of Thin Provisioning I would like to expand. I would recommend before migrating virtual machines that the “dead space” that will be left behind on the source datastore after the migration can be reclaimed by the use of the unmap primitive as part of VAAI.

Deduplication is a difficult one. The question is, will the “deduplication” process be as efficient after the migration as it was before the migration. Will it be able to deduplicate the same amount of data? There is always a chance that this is not the case… But than again, do you really care all that much about it when you are running out of disk space on your datastore or are exceeding your latency threshold? Those are very valid reasons to move a virtual disk as both can lead to degradation of service.

In an environment where replication is used care should be taken when balancing recommendations are applied. The reason for this being that the full virtual disk that is migrated will need to be replicated after the migration. This temporarily leads to an “unprotected state” and as such it is recommended to only migrate virtual disks which are protected during scheduled maintenance windows.

Auto-tiering arrays have been a hot debate lately. Not many seem to agree with my stance but up til today no one has managed to give me a great argument or explain to me exactly why I would not want to enable Storage DRS on auto-tiering solutions. Yes I fully understand that when I move a virtual machine from datastore A to datastore B the virtual machine will more than likely end up on relatively slow storage and the auto-tiering solution will need to optimize the placement again. However when you are running out of diskspace what would you prefer, down time or a temporary slow down? In the case of “I/O” balancing this is different and in a follow up post I will explain why this is not supported.

** This article is based on vSphere 5.0 information **

Punch Zeros!

I was just playing around with vSphere 5.0 and noticed something cool which I hadn’t noticed before. I logged in to the ESXi Shell and typed a command I used a lot in the past, vmkfstools, and I noticed an option called -K. (Just been informed that 4.1 has it as well, I never noticed it though… )

-K –punchzero
This option deallocates all zeroed out blocks and leaves only those blocks that were allocated previously and contain valid data. The resulting virtual disk is in thin format

This is one of those options which many have asked for as in order to re”thin” their disks it would normally require a Storage vMotion. Unfortunately though it only currently works when the virtual machine is powered off, but I guess that is just the next hurdle that needs to be taken.

vSphere 5.0: Profile-Driven Storage, what is it good for?

By now most of you heard about this new feature called Profile-Driven Storage that will be introduced with vSphere 5.0, but what is it good for? Some of you, depending on the size of the environment, currently have a nice long operational procedure to deploy virtual machines. The procedure usually contains gathering information about the requirements of the virtual machine’s disks, finding the right datastore to meet these requirements, deploy the virtual machine and occasionally check if the virtual machine’s disks are still placed correctly. This is what Profile-Driven Storage aims to solve.

Profile-Driven Storage, in the vCenter UI referred to as VM Storage Profiles, decrease the amount of administration required to properly deploy virtual machines by allowing for the creation of Profiles. These profiles typically list the requirements of storage and can be linked to a virtual machine. I know it all sounds a bit vague, let me visualize that:

In this scenario a virtual machine requires “Gold Storage”, now lets just assume for now that that means RAID-10 and Replicated. By linking the profile to this virtual machine it is possible to validate if the virtual machine is actually located on the right tier of storage. Now this profile can of course be linked to a virtual machine / virtual disk after it has been provisioned, but even more importantly it can be used during the provisioning of the virtual machine to ensure the user picks a datastore (cluster) which is compatible with the requirements! Just check the following screenshot of what that would look like:

Now you might wonder where this storage tier comes from, this is a VM Storage Profile containing storage capabilities provided by:

  • VASA aka vSphere Storage APIs – Storage Awareness
  • User defined capabilities

User defined capabilities are fairly simple to explain, the profile you create (gold / silver / bronze) will be linked to a User Defined “tag” you define on a datastore. For instance you could tag a datastore as “RAID-10″. When would you do this? Well typically when your storage vendor doesn’t offer a Storage Provider for VASA (yet). That takes us to the second method of selecting storage capabilities for your VM Storage Profile, VASA. VASA is a new “API” which enables you to see the characteristics of a datastore through vCenter. With characteristics I am referring to things like: raid level, de-duplication, replication etc. You know what, my a step-by-step guide makes it clear:

  • Go to VM Storage Profiles
  • Create a VM Storage Profile
  • Provide a Name
  • Select the correct Capabilities
  • Finish the creation
  • Create a new VM and select the correct VM Storage Profile, note that only 1 datastore is compatible
  • After creation you can easily check if it is compliant or not by going to the VMs summary tab

A couple of simple initial steps as you can clearly see, but a huge help when provisioning virtual machines and when validating storage / vm requirements!

vSphere 5.0: Storage DRS introduction

Storage DRS is a brand new feature of vSphere 5.0. It has been one of my focus areas for the last 6 months and probably one of the coolest features of vSphere 5.0. Storage DRS enables you to aggregate datastores in to a single object, called a datastore cluster. This new object is what you will be managing from now on. Storage DRS enables smart placement of virtual machines based on utilized diskspace, latency and LUN performance capabilities. In other words, when you create a new virtual machine you will select a Datastore Cluster instead of a Datastore and Storage DRS will place the virtual machine on one of the datastores in that datastore cluster. This is where the strength lies of Storage DRS, reducing operational effort associated with provisioning of virtual machines…

But that’s not all there is, Storage DRS is a lot more than just initial placement… lets sum the core functionality of Storage DRS up:

  1. Initial Placement
  2. Migration Recommendations (Manual / Fully Automated)
  3. Affinity Rules
  4. Maintenance Mode

These in my opinion are the 4 core pieces of functionality that Storage DRS provides. Initial placement as stated will reduce the amount of operational effort required to provision virtual machines. Storage DRS will figure out which datastore it should be placed on, no need anymore to manually monitor each datastore and figure out which one has the most available diskspace and relative low latency. On top of that SDRS also provides Migration Recommendations if and when thresholds are exceeded, it can generate them (manual mode) or generate and apply them (fully automated mode). These thresholds are utilized disk space(80%) and latency (15ms). This helps preventing bottlenecks in terms of disk space and hot spots in terms of latency.

Affinity Rules and Maintenance Mode are very similar to what DRS offers today. You have the ability to split disks and virtual machines with Affinity Rules, or keep them together. With Maintenance Mode it will be very easy to migrate to new LUNs or to do planned maintenance on a volume, couple of clicks and all VMs will be moved off.

Once again I would like to stress that although the Migration Recommendations (especially in Fully Automated mode) sound really sexy, and it is, it will more than likely be the Initial Placement recommendations where you will benefit the most. More technical information will follow soon here and on frankdenneman.nl