Someone asked me on the VMTN Communities and it’s something I had not looked into yet. The question was around VMFS block sizes and the way it could potentially have its effect on the size of a backup which uses CBT. The assumption was made that CBT on a 1MB block size VMFS volume uses 1MB blocks and on an 8MB block size VMFS volume uses 8MB blocks. This is not the case.

So what’s the size of the block that CBT refers to? Good question I’ve asked around and the answer is that it’s not a specific size but it has a variable size. The block always starts with 64KB and the bigger the VMDK becomes the bigger the blocks become.

Just for the sake of it:

• CBT is on a per VMDK level and not on a VMFS level.
• CBT has variable block sizes which are dictated by the size of the VMDK.
• CBT is a feature that lives within the VMKernel and not within VMFS.
• CBT is a FS Filter as shown in the VMworld slide below

NOTE The VSS component gets installed by default when you do a fresh installation of VMware Tools shipped with ESX Server 3.5 Update 2. If you upgrade from an earlier version, you need to install the VSS component manually.

Please be aware that when you want to use VSS on your existing environment you will need to manually upgrade your version of VMware Tools and select the VSS driver. For Windows 2008 and Vista the VSS driver is not installed by default even when doing a clean install this means that you will need to manually add the driver when VSS based backups is a requirement.

ghettoVCBg2

11/17/09 – The following enhancements and fixes have been implemented in this release of ghettoVCBg2. Special thanks goes out to Gerhard Ostermann for assisting with some of the logic in the ghettoVCBg2 script and the rest of the ghettoVCBg2 BETA testers. Thanks for everyones time and comments to make this script better!

Enhancements:

• Email log support
• Include/exclude specific VMDK(s)
• Additional logging + dry run mode

Fixes:

• Independent disk aware
• Large VMDK backups

Original script, but updated with new features and a bug fix:

ghettoVCB

11/17/09 – The following enhancements and fixes have been implemented in this release of ghettoVCB. Special thanks goes out to all the ghettoVCB BETA testers for providing time and their environments to test features/fixes of the new script!

Enhancements:

• Individual VM backup policy
• Include/exclude specific VMDK(s)
• Logging to file
• Timeout variables
• Configur snapshot memory/quiesce
• Adapter format
• Additional logging + dryrun mode
• Support for both physical/virtual RDMs

Fixes:

• Independent disk aware

Version 1.0.2
Build Number 188925
Release Date 2009/09/09

This releases fixes a couple of known issues:

• Various Integrity Check Issues
  Under certain circumstances, integrity checks reported damaged restore points and cannot load session errors. For example, such problems might be reported if:
  • A combination of simultaneous overlapping backups and integrity checks are started.
  • A backup is stopped before completion because the backup window closes. In such a case, the deduplication store records transactions, but the closing of the backup window prevents recording the transaction to the catalog.

  When integrity checks failed in such cases, Data Recovery would mark restore points as damaged or report that the backup session could not be found. Data Recovery integrity check now handles these conditions properly, so these problems no longer occur.

• Connections Using Alternate Ports not Supported
  By default, connections to vCenter Server use port 443. If vCenter Server is configured to use an alternate port, Data Recovery continued to attempt to connect using the default port. This caused the Data Recovery plug-in to report authentication failures when attempting to connect to the Data Recovery appliance. Alternate vCenter Server port configurations are now supported.
• Multiple VMDKs with the Same Name not Handled Properly
  A virtual machine can have multiple VMDK files with the same name that are stored on different LUNs. In such a case, Data Recovery would only restore one of the disks. Data Recovery now restores all disks.

You can find the full release notes here.

Latest Released Version: 1.0.1 | 07/09/09 | 176771

You can find the release notes here. I want to highlight the following resolved issue as it has been discussed by several bloggers(eric sloof, scott lowe):

Large Temporary Files Removed as Expected
Data Recovery modifies virtual machines’ vmdk files’ settings so a snapshot can be created for backup purposes. In the past, after the backup has been created, the vmdk file’s settings was sometimes left configured for snapshots even after the backup was complete. This led to these virtual machines being left in snapshot mode while accumulating snapshots that were undetected by vSphere Client. This process has been redesigned so that these temporary files are no longer be left behind. In previous versions of Data Recovery, this issue can be resolved by following the process described in the knowledge base article titled “Delete ddb.delete entries and snapshots left behind by Vmware Data Recovery”.

1. Create a snapshot of disk(s)
2. Hot add disk(s) to Data Recovery appliance
3. Create hashes of (hopefully variable) blocks
4. Read data of changed blocks if previous backup exists
5. Dedupe(using variable chunk sizes) and create SHA1 hash for index
6. Store data (possibly encrypted)
7. Remove hot add disk(s)
8. Remove snapshot

Compared to the current VCB installable and current feature set VI 3.5 offers this is a huge enhancement. (VMware vCenter Data Recovery will be part of the vSphere products.) Creating deduplicated back ups of only the changed blocks based on variable chunk sizes will give every user the opportunity to have a decent backup scheme. VMware vCenter Data Recovery utilizes the new VMware Consolidated Backup API by the way. For those afraid that the dedupe datastore gets corrupted an automated short integrity check is performed once a day and a thorough integrity check once a week.

Keep in mind that not only VMware will be able to utilize these new features. Because VCB is changed to an API a much tighter integration with 3rd party backup tools can be expected in the near future!

I would love to get my hands on a beta version of the product as soon as it’s available to play around with it some more and tell you more about the rich feature set this product will have. Unfortunately it’s not available yet and you will all have to wait, but I will keep you posted.

The VMTN Experts session was actually, in my opinion, pretty good today. We had way more people coming over that were asking questions or just came over to have a chat with one of the Experts! With the vExperts being announced this morning there was a good atmosphere, even Statler and Waldorf euuuh Boche and Laverick were having fun. For those on twitter and those who have been following the story about VMDougs snuggie, check the video Gabe published, lol.

Now today I did one of the labs together with Eric Sloof, Data Recovery. There are a couple of write-ups on the product for instance this one by VM-Aware so I’m going to write about all the features and scheduling possibilities. One thing I’ve noticed no one writing about is that the Appliance actually uses the hot-add feature which is part of VCB. So basically what happens when you do a backup:

1. Create a snapshot of disk(s)
2. Hot add disk(s) to Data Recovery appliance
3. Create hashes of (hopefully variable) blocks
4. Read data of changed blocks
5. Dedupe(using variable chunk sizes) and create hash
6. Store data
7. Remove hot add disk(s)
8. Remove snapshot

So this is what I’m guessing will happen cause I actually haven’t seen the documentation, but it makes sense in my opinion.. The cool thing about this way of running backups is that doesn’t really matter if you do a full backup or do an incremental/differential it’s always small because of the deduplication. Full backup it is! Although some blogs aren’t sure yet, it will contain file level restore in the future!

By using the hot add mode it’s a LAN free backup, all I/O is being handled by the ESX Server I/O stack. This will however cause overhead on the ESX Server so you might not want to start 20 backups on the same host. Also keep in mind that the deduplication will be rather CPU intensive so this might also slow down the process if you run multiple backups. The deduplication is “inline” by the way which means that the data will be deduped before it will be written to disk.

Now there’s more to write about, and especially about the Cloud Plugin that has been demoed today… But I really need to get ready for the VMworld party called “Cloud9″ this year! See you guys in a few minutes and for those that didn’t come to Cannes, please join John Troyer tonight(wednesday) at the weekly podcast and remind him he’s really missing out on this!

In addition to the ESXi backup through VCB that was introduced with version 2.0, Veeam Backup 3.0 now supports ESXi backup without VCB. Veeam Backup is the only VMware backup solution that lets you backup and restore virtual machines running on all existing editions of ESXi, including ESXi free.

Now head over to Veeam and download your trial and start doing full VM backups. Veaam Backup also provides you with a file level restore:

Veeam’s fast file-level recovery feature allows you to restore individual files from your image-level backups and replicas in seconds, without having to extract the full VM image to your local drive.

Just a tip, I would suggest to do a “zero-out” before running the full VM backup!

This document contains a script and a list file. The list file should contain all the VM’s that need to be backed up. You can hook your ESXi box up to an NFS share and backup your VM’s to the NFS share. (NFS share could be a Windows Server for instance) You can set several variables to adjust the script to your needs:

• Backup location
• Sparse files(2GB chunks)
• Backup Rotation
• Directory naming scheme
• NFS mount options

I haven’t personally tested the script but looking at the feedback it should work great. I think this is definitely worth checking out. So if you need a backup solution for your ESXi host and don’t want to spend money on 3rd party tools, check this one.