Differencing Hard Disk - Hyper-V

I came across this option when creating a new had drive in Hyper-V.

Has anybody ever used one - is it analogous to e.g. a checkpoint, or an incremental image backup in Macrium Reflect?

When and how would you use it?



Edit - been reading up on it and they look quite cool for a test environment.

It is rather like a Macrium Differential image ie it contains all the changes from base version.

So you could have a base install, do a cumulative update in a differencing drive, test it, and if ok, merge the results (to original or new drive).

If you decide to discard the differencing drive you simply delete it and set the base drive as the drive in hyper-v.

Of course, you cannot modify the base drive and keep a differencing drive based on earlier version of base image.

One cool thing I see is ability to have a base image, then sysprep to a differencing drive, and then merge the base and differencing drive to a new drive.

One weakness dynamics vhds have is they grow as you do updates, but even if you delete stuff using disk cleanup, you cannot physically recover the space even if you use powershell shrink commands.

The usual solution is to copy partitions to a new vhd using tools like Minitool Partition Wizard or Macrium Reflect adjusting sizes to recover space.

I am hoping that if you merge a base and differencing drive to a new one, space is optimally recovered. My project this week :-D

cereberus said:

I came across this option when creating a new had drive in Hyper-V.

Has anybody ever used one - is it analogous to e.g. a checkpoint, or an incremental image backup in Macrium Reflect?

When and how would you use it?

Decided to show how I use differencing disks, and how I benefit from them. Apparently, as your edits show, you already found answers to your questions, but maybe this helps other members to understand the concept.

Some facts first. Differencing disks means that a VM uses a so called child VHD for an existing parent VHD. The parent VHD remains intact, in fact it can even be write protected. All changes made to VM are saved to child VHD. In some sense you could call it "checkpoint on steroids"; create a new VM using a child VHD, test your stuff, when done delete the VM and its child VHD to undo changes. Parent VHD is still there, repeat the process to test something else.

A differencing disk is always dynamically expanding, it can't be created as fixed size VHD. It's maximum size cannot be set by user, it can expand endlessly until the drive where it is stored has no more free space for it to expand.

One reason I use differencing disks is to save storage space on my dedicated Hyper-V partition. I use it mainly for two slightly different scenarios. Another good reason is how fast it is to create new virtual machines.

 Scenario 1: Different W10 versions

I installed Windows 10 version 1803 on a Hyper-V VM, When done, I simply deleted the VM, leaving only its VHDX file on my Hyper-V VM partition. I renamed that VHDX file as W10 1803 Base.

Next, I created three differencing virtual hard disks, naming them W10 Insider Preview Fast, W10 Insider Preview Skippy, and W10 version 1803. All three are using the same base, parent VHD W10 1803 Base.

In Hyper-V Manager I then created three new virtual machines, each of them using one of those three child disks. Booted up VM W10 Insider Preview Fast and upgraded it to latest Fast Ring build, booted up VM W10 Insider Preview Skippy and upgraded it to latest Skip Ahead build, and booted up VM W10 version 1803 and have been updating it with all cumulative and other updates for last official release. In a month to six weeks, when version 1809 will be released, that last mentioned VM will be upgraded and renamed to W10 version 1809.


(Click to enlarge.)

 Scenario 2: Different domain controllers

I clean installed Windows Server 2016 on a VM and use its VHDX file as parent disk. I have created three differencing disks and use them for virtual machines for two different Server 2016 local domains / domain controllers, and use third one to test Server 2019 Preview (upgraded from Server 2016), using it for third local domain.

Much faster to setup than to install server OS three times, a lot of HDD space saved.

How do I save space? Let's say that I had those three virtual machines mentioned in Scenario 1 above done traditionally using normal dynamically expanding virtual hard disks. The size of their VHDX files would be at the moment, depending on how many upgrades the public release, Fast Ring and Skippy virtual machines would have gone through around 25 to 30 GB each.

Using one parent VHD, it's size always remains at around 10 GB. It will not be changed, updated or upgraded, nothing will be saved on it. All three child VHDs, when VMs using them have been created only use 4 MB to start with. Of course these files will grow, my version 1803 VM child disk is now over 12 GB (it started its life as version 1709, was later upgraded to 1803), both Fast and Skippy already have over 15 GB child disks as they have been through a few upgrades.

All together, my parent (base) disk plus all three child disks, the VHDX files, use currently a total of 53 GB of storage space. Having done the same three virtual machines using normal dynamically expanding virtual hard disks, they would at the moment consume at least 80 GB of HDD space.

That's one way to use differencing disks.

Kari

Kari said:

Decided to show how I use differencing disks, and how I benefit from them. Apparently, as your edits show, you already found answers to your questions, but maybe this helps other members to understand the concept.

Some facts first. Differencing disks means that a VM uses a so called child VHD for an existing parent VHD. The parent VHD remains intact, in fact it can even be write protected. All changes made to VM are saved to child VHD. In some sense you could call it "checkpoint on steroids"; create a new VM using a child VHD, test your stuff, when done delete the VM and its child VHD to undo changes. Parent VHD is still there, repeat the process to test something else.

A differencing disk is always dynamically expanding, it can't be created as fixed size VHD. It's maximum size cannot be set by user, it can expand endlessly until the drive where it is stored has no more free space for it to expand.

One reason I use differencing disks is to save storage space on my dedicated Hyper-V partition. I use it mainly for two slightly different scenarios. Another good reason is how fast it is to create new virtual machines.

 Scenario 1: Different W10 versions

I installed Windows 10 version 1803 on a Hyper-V VM, When done, I simply deleted the VM, leaving only its VHDX file on my Hyper-V VM partition. I renamed that VHDX file as W10 1803 Base.

Next, I created three differencing virtual hard disks, naming them W10 Insider Preview Fast, W10 Insider Preview Skippy, and W10 version 1803. All three are using the same base, parent VHD W10 1803 Base.

In Hyper-V Manager I then created three new virtual machines, each of them using one of those three child disks. Booted up VM W10 Insider Preview Fast and upgraded it to latest Fast Ring build, booted up VM W10 Insider Preview Skippy and upgraded it to latest Skip Ahead build, and booted up VM W10 version 1803 and have been updating it with all cumulative and other updates for last official release. In a month to six weeks, when version 1809 will be released, that last mentioned VM will be upgraded and renamed to W10 version 1809.


(Click to enlarge.)

 Scenario 2: Different domain controllers

I have a clean installed Windows Server 2016 and use its VHDX file as parent disk. I have created three differencing disks and use them for virtual machines for two different Server 2016 local domains / domain controllers, and use third one which I upgraded to Server 2019 Preview, using it for third local domain.

Much faster to setup than to install server OS three times, a lot of HDD space saved.

How do I save space? Let's say that I had those three virtual machines mentioned in Scenario 1 above done traditionally using normal dynamically expanding virtual hard disks. The size of their VHDX files would be at the moment, depending on how many upgrades the public release, Fast Ring and Skippy virtual machines would have gone through around 25 to 30 GB each.

Using one parent VHD, it's size always remains at around 10 GB. It will not be changed, updated or upgraded, nothing will be saved on it. All three child VHDs, when VMs using them have been created only use 4 MB to start with. Of course these files will grow, my version 1803 VM child disk is now over 12 GB (it started its life as version 1709, was later upgraded to 1803), both Fast and Skippy already have over 15 GB child disks as they have been through a few upgrades.

All together, my parent (base) disk plus all three child disks, the VHDX files, use currently a total of 53 GB of storage space. Having done the same three virtual machines using normal dynamically expanding virtual hard disks, they would at the moment consume at least 80 GB of HDD space.

That's one way to use differencing disks.

Kari

Thanks Kari, as you say I got the gist of it. One thing is evident (and articles said this) is that you do need to do it really on an ssd as hdds are not really fast enough.

May I suggest that this might make an interesting topic at the next TF Live session?

If I understand the articles I read, you can do similar with vhds mounted on host but you have to use powershell commands rather than the Hyper-V gui to create differencing drives etc.

I like the idea of this as I sometimes use dism to install Insider versions to a dual boot vhd, and it would be nice to use differencing drives there as well. It is a pity you cannot upgrade W10 builds in a vhd on a host pc (easy enough to use Hyper-V).

Well this is cool - you can link differencing drives in a chain e.g. Parent - child - grandchild etc.

cereberus said:

Well this is cool - you can link differencing drives in a chain e.g. Parent - child - grandchild etc.

Yes, that is cool.

About something you mentioned in your previous post: I'll check something later when I have time, but I believe there's a quite easy way to use differencing disks in native boot.

Kari

I followed later part of this guide which says mount differencing drive and use bcdboot to add a boot entry but it would not let me mount it as a drive saying it was in use.

Windows 8 Boot to Differencing VHD


PS - as we know, it is a good idea to create base vhd in Hyper-V first so it has the hidden efi partition as well so vhd can be used on hyper-V as well.

jimbo45 said:

Hi there

thanks guys --interesting discussion between the pair of you --great stuff.

Interesting -- will have to try this myself.

I am intrigued with this concept - especially with fast ring and skippy builds coming out thick and fast recently.

Cheers
jimbo

Remember if you use the parent, it kills the child and you will be guilty of filicide :-D.

You can only use any differencing drives at the bottom of the chain to keep chain intact.

If you want to use builds higher up, use a copy.

Did a test. Had a base image of 1803 (approx 12 GB, upgraded to 1809 in the differencing drive (17GB but included Windows.old).

Did disk cleanup on Differencing drive, and then used merge option to merge base drive and differencing drive, to a new merged drive, and merged drive was 27GB!

It should only be around 13-14 GB maximum. Obviously, the merging algorithm is really space inefficient.

Back to my good old space efficient minitool method!