Yellow Bricks

by Duncan Epping

backup

in the ghetto….

· ·

William Lam just updated two of his most popular scripts. If you haven’t looked at them yet, make sure you do as they are worth it. ghettoVCB(g2) enables the backup of virtual machines residing on either an ESX or ESXi host. ghettoVCBg2 is a complete rewritten and enhanced version of ghettoVCB or as William puts it “harder, better, faster, stronger”.

ghettoVCBg2

11/17/09 – The following enhancements and fixes have been implemented in this release of ghettoVCBg2. Special thanks goes out to Gerhard Ostermann for assisting with some of the logic in the ghettoVCBg2 script and the rest of the ghettoVCBg2 BETA testers. Thanks for everyones time and comments to make this script better!

Enhancements:

  • Email log support
  • Include/exclude specific VMDK(s)
  • Additional logging + dry run mode

Fixes:

  • Independent disk aware
  • Large VMDK backups

Original script, but updated with new features and a bug fix:

ghettoVCB

11/17/09 – The following enhancements and fixes have been implemented in this release of ghettoVCB. Special thanks goes out to all the ghettoVCB BETA testers for providing time and their environments to test features/fixes of the new script!

Enhancements:

  • Individual VM backup policy
  • Include/exclude specific VMDK(s)
  • Logging to file
  • Timeout variables
  • Configur snapshot memory/quiesce
  • Adapter format
  • Additional logging + dryrun mode
  • Support for both physical/virtual RDMs

Fixes:

  • Independent disk aware

VMware Data Recovery 1.0.2

· ·

VMware just released a brand new version of VMware Data Recovery.

Version 1.0.2
Build Number 188925
Release Date 2009/09/09

This releases fixes a couple of known issues:

  • Various Integrity Check Issues
    Under certain circumstances, integrity checks reported damaged restore points and cannot load session errors. For example, such problems might be reported if:

    • A combination of simultaneous overlapping backups and integrity checks are started.
    • A backup is stopped before completion because the backup window closes. In such a case, the deduplication store records transactions, but the closing of the backup window prevents recording the transaction to the catalog.

    When integrity checks failed in such cases, Data Recovery would mark restore points as damaged or report that the backup session could not be found. Data Recovery integrity check now handles these conditions properly, so these problems no longer occur.

  • Connections Using Alternate Ports not Supported
    By default, connections to vCenter Server use port 443. If vCenter Server is configured to use an alternate port, Data Recovery continued to attempt to connect using the default port. This caused the Data Recovery plug-in to report authentication failures when attempting to connect to the Data Recovery appliance. Alternate vCenter Server port configurations are now supported.

  • Multiple VMDKs with the Same Name not Handled Properly
    A virtual machine can have multiple VMDK files with the same name that are stored on different LUNs. In such a case, Data Recovery would only restore one of the disks. Data Recovery now restores all disks.

You can find the full release notes here.

VMware Data Recovery 1.0.1 released!

· ·

Although the VMware download section has not been updated VMware Data Recovery 1.0.1 has been officially released. Click on VMware Data Recovery 1.0 in the download section and you will be presented with the following:

Latest Released Version: 1.0.1 | 07/09/09 | 176771

You can find the release notes here. I want to highlight the following resolved issue as it has been discussed by several bloggers(eric sloof, scott lowe):

Large Temporary Files Removed as Expected
Data Recovery modifies virtual machines’ vmdk files’ settings so a snapshot can be created for backup purposes. In the past, after the backup has been created, the vmdk file’s settings was sometimes left configured for snapshots even after the backup was complete. This led to these virtual machines being left in snapshot mode while accumulating snapshots that were undetected by vSphere Client. This process has been redesigned so that these temporary files are no longer be left behind. In previous versions of Data Recovery, this issue can be resolved by following the process described in the knowledge base article titled “Delete ddb.delete entries and snapshots left behind by Vmware Data Recovery”.

VMware vCenter Data Recovery

· ·

During VMworld I quickly wrote down the steps that VMware vCenter Data Recovery takes when backing up VMs. I added one step which isn’t unimportant, changed block tracking(4):

  1. Create a snapshot of disk(s)
  2. Hot add disk(s) to Data Recovery appliance
  3. Create hashes of (hopefully variable) blocks
  4. Read data of changed blocks if previous backup exists
  5. Dedupe(using variable chunk sizes) and create SHA1 hash for index
  6. Store data (possibly encrypted)
  7. Remove hot add disk(s)
  8. Remove snapshot

Compared to the current VCB installable and current feature set VI 3.5 offers this is a huge enhancement. (VMware vCenter Data Recovery will be part of the vSphere products.) Creating deduplicated back ups of only the changed blocks based on variable chunk sizes will give every user the opportunity to have a decent backup scheme. VMware vCenter Data Recovery utilizes the new VMware Consolidated Backup API by the way. For those afraid that the dedupe datastore gets corrupted an automated short integrity check is performed once a day and a thorough integrity check once a week.

Keep in mind that not only VMware will be able to utilize these new features. Because VCB is changed to an API a much tighter integration with 3rd party backup tools can be expected in the near future!

I would love to get my hands on a beta version of the product as soon as it’s available to play around with it some more and tell you more about the rich feature set this product will have. Unfortunately it’s not available yet and you will all have to wait, but I will keep you posted.

VMworld day 2 – Random stuff and Data recovery

· ·

What a day again, and it’s actually not finished cause within a couple of minutes the VMworld Europe 2009 party will start.

The VMTN Experts session was actually, in my opinion, pretty good today. We had way more people coming over that were asking questions or just came over to have a chat with one of the Experts! With the vExperts being announced this morning there was a good atmosphere, even Statler and Waldorf euuuh Boche and Laverick were having fun. For those on twitter and those who have been following the story about VMDougs snuggie, check the video Gabe published, lol.

Now today I did one of the labs together with Eric Sloof, Data Recovery. There are a couple of write-ups on the product for instance this one by VM-Aware so I’m going to write about all the features and scheduling possibilities. One thing I’ve noticed no one writing about is that the Appliance actually uses the hot-add feature which is part of VCB. So basically what happens when you do a backup:

  1. Create a snapshot of disk(s)
  2. Hot add disk(s) to Data Recovery appliance
  3. Create hashes of (hopefully variable) blocks
  4. Read data of changed blocks
  5. Dedupe(using variable chunk sizes) and create hash
  6. Store data
  7. Remove hot add disk(s)
  8. Remove snapshot

    So this is what I’m guessing will happen cause I actually haven’t seen the documentation, but it makes sense in my opinion.. The cool thing about this way of running backups is that doesn’t really matter if you do a full backup or do an incremental/differential it’s always small because of the deduplication. Full backup it is! Although some blogs aren’t sure yet, it will contain file level restore in the future!

    By using the hot add mode it’s a LAN free backup, all I/O is being handled by the ESX Server I/O stack. This will however cause overhead on the ESX Server so you might not want to start 20 backups on the same host. Also keep in mind that the deduplication will be rather CPU intensive so this might also slow down the process if you run multiple backups. The deduplication is “inline” by the way which means that the data will be deduped before it will be written to disk.

    Now there’s more to write about, and especially about the Cloud Plugin that has been demoed today… But I really need to get ready for the VMworld party called “Cloud9” this year! See you guys in a few minutes 🙂 and for those that didn’t come to Cannes, please join John Troyer tonight(wednesday) at the weekly podcast and remind him he’s really missing out on this! 😛