Backup and compare with robocopy not working  

When you turn on logging, the status will no longer be displayed to the screen unless you add the switch "/TEE". That is why you only see the message "Log File : C:\compare.txt" and no other status while the copy is in progress.

Also, make sure that you are running robocopy from an elevated command prompt.

While I've not tried this with the Windows directory, it might be worth also throwing in the "/B" switch to backup files in backup mode. This may help since many files will likely be in use or locked as Samuria noted.

Be aware also that your Windows folder is a "living breathing thing" at least in the respect that it is always changing. Temporary files are created and destroyed, other files are modified. I wouldn't expect the entire directory structure to be the same from one minute to the next, let alone in the time it takes to copy that whole thing

One more thing to add: I was just trying this, and even with the /B I got an error trying to copy one of the registry hives. I added a "/R:0" to prevent robocopy from doing the default of 1 million retries . So far, it is proceeding on. We'll see how close I get to having the same size and number of files / folders.

Done with my copy. Interestingly, my destination is about 400MB larger than the source and has about 2,000 more files and 2,000 more folders than the source.

If I run the same robocopy command again, I see a bunch of changed and newer files (I added a /NDL on subsequent runs so that it would only list files that have changed and not every one of the tens of thousands of directories).

Will try to research further.

Have not found anything really earthshattering, but from what I do know I would think that the proper way to do this would be something like this:

1) Create a shadow copy of the drive first.
2) Begin your backup of that shadow copy.
3) When done, do your comparison with the shadow copy.

Note that this is the technique used by backup programs such as Macrium Reflect.

The whole point of a shadow copy is that you can backup files that would normally be in use and also so that things are not changing on you while the backup is being performed. Once the shadow copy is created, you have a consistent set of data that will not change until that shadow copy is destroyed.

As an example, I have a drive with an excel spreadsheet. I start a backup that begins with the creation of a shadow copy. In the middle of the backup, a user deletes that spreadsheet and the backup has not yet gotten to the folder where it resides. It doesn't matter because the file is still preserved in the shadow and so it will be backed up.

That's my thinking on the matter, but if anyone has other thoughts, please do say. I'm interested in learning something here

More info can be found here:

Backup Internals: What is VSS, how does it work and why do we use it? | by Macrium Software | Macrium Software.

hsehestedt said:

Have not found anything really earthshattering, but from what I do know I would think that the proper way to do this would be something like this:

1) Create a shadow copy of the drive first.
2) Begin your backup of that shadow copy.
3) When done, do your comparison with the shadow copy.

Note that this is the technique used by backup programs such as Macrium Reflect.

The whole point of a shadow copy is that you can backup files that would normally be in use and also so that things are not changing on you while the backup is being performed. Once the shadow copy is created, you have a consistent set of data that will not change until that shadow copy is destroyed.

As an example, I have a drive with an excel spreadsheet. I start a backup that begins with the creation of a shadow copy. In the middle of the backup, a user deletes that spreadsheet and the backup has not yet gotten to the folder where it resides. It doesn't matter because the file is still preserved in the shadow and so it will be backed up.

That's my thinking on the matter, but if anyone has other thoughts, please do say. I'm interested in learning something here

More info can be found here:

Backup Internals: What is VSS, how does it work and why do we use it? | by Macrium Software | Macrium Software.

I didn't mention that none of the 2 drives involved (source and target) is the boot drive I'm using, as I thought the E:\ and F:\ paths would be telling (sorry I wasn't more clear).
I'll try again with the /b (both on the backup and on the comparison test) and come back to you.
Thanks a lot for your answers.

- - - Updated - - -

Samuria said:

The problem is both permissions and use a lot of files may be locked as they are in use and some may change while windows is running so your never going to get an exact match just copying

I didn't mention that none of the 2 drives involved (source and target) is the boot drive I'm using, as I thought the E:\ and F:\ paths would be telling (sorry I wasn't more clear).
I'll try again with the /b (both on the backup and on the comparison test) and come back to you.
Thanks a lot for your answer

- - - Updated - - -

Try3 said:

Windows' count of the in C:\Windows files while Windows is running & its estimation of their size are known to be useless.
- That's why tools such as Treesize as recommended for people investigating issues within C:\Windows.

So this thread seems to be about comparing the incorrect count of files with another count of files derived from that incorrect count of files.

No correct answer can possibly come from this.

Antonio - If your aim is to get a full copy of the C:\Windows folder then boot from an InstallUSB to do it. You can still use RoboCopy.
- Size comparisons might still be rubbish. C:\Windows includes some items [such as many within the component store] that appear to be files but are really hard-wired redirections to files that are elsewhere within C:\Windows. I have no idea how RoboCopy will cope with them.
- I think you'd be better off making a system image.

Denis

I tried with Treesize and it gives me exactly the same result as File Explorer. Then I compared source and backup with WinMerge and the result seems to indicate that neither File Explorer nor Winmerge are accessing some files and folders in the source directory.

My initial question is partly solved by adding /B to the robocopy comparison command:
robocopy F:\Windows E:\roboRubi\Windowsz /L /NJH /NJS /NP /NS /B /log:C:\compare.txt
...but I don't understand the output file, it seems to show all of the file transfers instead of only the differences? what I need is the differences.

You forgot to add the /NDL switch I noted previously. Without it, robocopy will list every blasted directory whether there are any differences or not. Add the /NDL and only file differences will be shown. Note that the full path for each file will still be shown. Also, missing and extra directories will still be shown. The /NDL will merely suppress the listing of every directory that is not relevant to actual differences.

I'm still researching this:
Regarding how File Explorer counts files and folders:

File Explorer count of files and folders (right-click a folder > properties), What is counted and what is not:
I did an extensive test with a folder that contained all kind of files and folders: file shortcuts, folder shortcuts, system files, hidden files, file hardlinks, file symlinks, offline files (Onedrive files), Junction Points and Directory Symlinks.

FOLDERS: Junction Points and Directory Symlinks aren't counted. Folder shortcuts aren't counted (these are counted as files!). Hidden folders, system folders etc, all are included in the count.

FILES: all types were included in the count: file shortcuts, FOLDER SHORTCUTS (YES, THESE ARE COUNTED AS FILES!), system files, hidden files, file hardlinks, file symlinks, offline files (Onedrive files). Again: FOLDER SHORTCUTS ARE COUNTED AS FILES.

Stuff of any kind within folder shortcuts, within JPoints, and within folder symlinks IS NOT COUNTED as neither files nor folders, same with folder shortcuts within folder shortcuts: not counted.


Windows 10 20H2
Tested on NTFS folder.