Today I encountered an old misunderstood principle again. A customer had created several snapshots on a virtual machine. Several… well to be exact 15. All snapshots were larger than 20GB. When the VMFS volume, on which this VM was located, ran out of diskspace he decided to use the button “Delete All”, but within a couple of minutes the VMFS volume ran out of diskspace again. What happened?
Situation:
Snapshot 1 – 20GB
Snapshot 2 – 10GB
Snapshot 3 – 30GB
When you choose “delete all” the following will happen:
- Snapshot 2 will grow to 40GB at most
- Snapshot 1 will grow to 60GB at most
- Snapshot 1 will be committed to the original VMDK
- All snapshot files are deleted
In other words: Snapshot 3 is merged into Snapshot 2, Snapshot 2 is merged into Snapshot 1, Snapshot 1 is merged into the original flat.vmdk and afterwards all snapshot files are deleted. This means that if you want to delete all snapshots at once you will need around 130GB of free diskspace. So think twice when you press the “delete all” button.
vSphere 4.0 Quick Start Guide
Goede TIP!
[...] Delete all snapshots: For those end users that don’t work with snapshots, this article is a must read. [...]
[...] Another thing that people don’t know is the way “delete all” works, but I’ve already outlined that a while ago in a blog. [...]
[...] Yellow Bricks has a good little write about problems you could run into if you use a lot of snaps then delete all snapshots. Make sure you have a lot of extra disk space when you do it. [...]
[...] some quick Googling, I found Delete all Snapshots which made me hold off on it until I had some more information about our snapshots. This number [...]
You know you IT types are so out of touch with reality.
The reasons for searching databases like yours is to recover from program defects.
If you don’t want to give an answer, stop posting.
How helpful is it to say “think twice”.
By the way i remeber also that deleting snapshot process (committing changes) in VC console, viewed below in status bar, generate a timeout error when it arrives at 95%.
This is not a problem, because on side Host the process continue its works and finish after sometimes (it depends on how many GB are going to commit.. maybe hours..). So don’t worry!!! Be patient!!!
[...] it didn’t work. Now the machine won’t stay booted.I remembered reading something from Yellow-Bricks about disk space and snapshots. Oops. Since this VM was on an ESXi host, there was no service console commands to commit the [...]
The author probably assumed a certain level of intelligence in his readers.
By ‘think twice’ he clearly means to think about whether you have enough free space on the volume to commit the snapshots.
Correct, think twice before you start deleting… My Article wasn’t about solving a problem, it was about explaining what happens in the background. It’s the way it’s been designed.
If you just start by deleting the oldest snapshot then you need no additional storage at all. It then just gets merged in the original vmdk-file.
Ok, so I don’t necessarily agree with oinkmaster… My VMs tend to be quite complicated, I keep the system drive’s VMDK on one datastore, along with the config files. I then tend to have several additional VMDK files that I keep on different VMFS datastores.
Example, if the VM is a database, then I have a database LUN that is a lot faster and has more expensive technology at the SAN level, and then the logfiles have their own partition at the VM level, which is an additional VMFS datastore on what we call “medium” LUNs, which are kind of in between the LUNs where we keep the OS vs LUNs where we keep the Databases…
So when I snapshot, even though the VM is associated with several LUNs, the delta is always written to the LUN/VMFS datastore where the VM’s config files are located. I have never seen this documented and had to learn this on my own, and it is possible I am just incorrect/crazy
So yes, if your VM has a single VMDK file and the config files are stored on the same VMFS datastore, then Oinkmaster is correct, the space you use on the VMFS will probably not change much (although if the delta file contains say a delete command for half of the space, then I imagine when the snapshot is applied it would actually shrink the VMDK…?)
But, in more complex scenarios, like where I have snapshotted my database VM and the delta for the LOGS vmdk has been written to a different datastore (the datastore where the VM’s config files are) then deleting/applying the snapshot most definitely causes the VMDK file to grow. And it can grow a lot.
So in this case, you should not only be aware of the VMDK growing when you delete a snapshot, you also need to make sure that the VMFS datastore where the configs are located for that VM has enough free space to house the delta. Leaving a snapshot for too long can not only hose the VM, but it you are overprovisioned on your datastore (and with snapshots you may end up being overprovisioned and not know it) you really need to be aware of the free space left on the datastore that houses the config files for that particular VM.
Oinkmaster is right, the nicer way to remove several snapshots is to remove the nearest to the Base Disk repeatedly. This doesn’t require additional space.
If the Snapshot Manager displays no snapshots… then the problem begins.
On http://vmutils.blogspot.com/ you can find an script that will tell you in advance how much space the VM would need to ‘delete all’.
However, better than solve a problem is to prevent it.
http://www.youtube.com/watch?v=gl0VmmKNhYk