Backup and compare with robocopy not working


  1. Posts : 163
    Windows 10
       #1

    Backup and compare with robocopy not working


    I've backed up the Windows folder from a Windows 10 boot drive with

    robocopy F:\Windows E:\roboRubi\Windowsz /mir

    Both source and target drives are attached to internal SATA ports. None of the two drives (source and target) is the boot drive I'm booted into. I'm using an elevated command prompt.

    I compared the two folders afterwards in File Explorer and found that, surprisingly, the backup folder has more files and overall size than the source folder (as shown in picture).

    I suspect both folders are really the same, but that Windows 10 may not be accessing all files in the source folder due to permission issues, I mean when I compare then in File Explorer?

    So I tried to compare both folders with

    robocopy F:\Windows E:\roboRubi\Windowsz /L /NJH /NJS /NP /NS /log:C:\compare.txt

    but this doesn't seem to work, it just instantly displays this path and does nothing:

    Log File : C:\compare.txt

    Is there a way to check this backup properly?
    I'd rather use robocopy than any other third party tool.

    Any help much welcome

    Windows 10 Pro 20H2.
    Backup and compare with robocopy not working-rzp1l.png
    Last edited by Frank15; 22 Apr 2021 at 16:18.
      My Computer


  2. Posts : 6,917
    windows 10
       #2

    The problem is both permissions and use a lot of files may be locked as they are in use and some may change while windows is running so your never going to get an exact match just copying
      My Computer


  3. Posts : 2,773
    Windows 11 Pro, 21H2
       #3

    When you turn on logging, the status will no longer be displayed to the screen unless you add the switch "/TEE". That is why you only see the message "Log File : C:\compare.txt" and no other status while the copy is in progress.

    Also, make sure that you are running robocopy from an elevated command prompt.

    While I've not tried this with the Windows directory, it might be worth also throwing in the "/B" switch to backup files in backup mode. This may help since many files will likely be in use or locked as Samuria noted.

    Be aware also that your Windows folder is a "living breathing thing" at least in the respect that it is always changing. Temporary files are created and destroyed, other files are modified. I wouldn't expect the entire directory structure to be the same from one minute to the next, let alone in the time it takes to copy that whole thing
      My Computers


  4. Posts : 2,773
    Windows 11 Pro, 21H2
       #4

    One more thing to add: I was just trying this, and even with the /B I got an error trying to copy one of the registry hives. I added a "/R:0" to prevent robocopy from doing the default of 1 million retries . So far, it is proceeding on. We'll see how close I get to having the same size and number of files / folders.
      My Computers


  5. Posts : 2,773
    Windows 11 Pro, 21H2
       #5

    Done with my copy. Interestingly, my destination is about 400MB larger than the source and has about 2,000 more files and 2,000 more folders than the source.

    If I run the same robocopy command again, I see a bunch of changed and newer files (I added a /NDL on subsequent runs so that it would only list files that have changed and not every one of the tens of thousands of directories).

    Will try to research further.
      My Computers


  6. Posts : 2,773
    Windows 11 Pro, 21H2
       #6

    Have not found anything really earthshattering, but from what I do know I would think that the proper way to do this would be something like this:

    1) Create a shadow copy of the drive first.
    2) Begin your backup of that shadow copy.
    3) When done, do your comparison with the shadow copy.

    Note that this is the technique used by backup programs such as Macrium Reflect.

    The whole point of a shadow copy is that you can backup files that would normally be in use and also so that things are not changing on you while the backup is being performed. Once the shadow copy is created, you have a consistent set of data that will not change until that shadow copy is destroyed.

    As an example, I have a drive with an excel spreadsheet. I start a backup that begins with the creation of a shadow copy. In the middle of the backup, a user deletes that spreadsheet and the backup has not yet gotten to the folder where it resides. It doesn't matter because the file is still preserved in the shadow and so it will be backed up.

    That's my thinking on the matter, but if anyone has other thoughts, please do say. I'm interested in learning something here

    More info can be found here:

    Backup Internals: What is VSS, how does it work and why do we use it? | by Macrium Software | Macrium Software.
      My Computers


  7. Posts : 11,391
    Windows 10 Home x64 Version 21H2 Build 19044.1706
       #7

    Windows' count of the in C:\Windows files while Windows is running & its estimation of their size are known to be useless.
    - That's why tools such as Treesize as recommended for people investigating issues within C:\Windows.

    So this thread seems to be about comparing the incorrect count of files with another count of files derived from that incorrect count of files.

    No correct answer can possibly come from this.

    Antonio - If your aim is to get a full copy of the C:\Windows folder then boot from an InstallUSB to do it. You can still use RoboCopy.
    - Size comparisons might still be rubbish. C:\Windows includes some items [such as many within the component store] that appear to be files but are really hard-wired redirections to files that are elsewhere within C:\Windows. I have no idea how RoboCopy will cope with them.
    - I think you'd be better off making a system image.

    Denis
      My Computer


  8. Posts : 163
    Windows 10
    Thread Starter
       #8

    hsehestedt said:
    Have not found anything really earthshattering, but from what I do know I would think that the proper way to do this would be something like this:

    1) Create a shadow copy of the drive first.
    2) Begin your backup of that shadow copy.
    3) When done, do your comparison with the shadow copy.

    Note that this is the technique used by backup programs such as Macrium Reflect.

    The whole point of a shadow copy is that you can backup files that would normally be in use and also so that things are not changing on you while the backup is being performed. Once the shadow copy is created, you have a consistent set of data that will not change until that shadow copy is destroyed.

    As an example, I have a drive with an excel spreadsheet. I start a backup that begins with the creation of a shadow copy. In the middle of the backup, a user deletes that spreadsheet and the backup has not yet gotten to the folder where it resides. It doesn't matter because the file is still preserved in the shadow and so it will be backed up.

    That's my thinking on the matter, but if anyone has other thoughts, please do say. I'm interested in learning something here

    More info can be found here:

    Backup Internals: What is VSS, how does it work and why do we use it? | by Macrium Software | Macrium Software.
    I didn't mention that none of the 2 drives involved (source and target) is the boot drive I'm using, as I thought the E:\ and F:\ paths would be telling (sorry I wasn't more clear).
    I'll try again with the /b (both on the backup and on the comparison test) and come back to you.
    Thanks a lot for your answers.

    - - - Updated - - -

    Samuria said:
    The problem is both permissions and use a lot of files may be locked as they are in use and some may change while windows is running so your never going to get an exact match just copying
    I didn't mention that none of the 2 drives involved (source and target) is the boot drive I'm using, as I thought the E:\ and F:\ paths would be telling (sorry I wasn't more clear).
    I'll try again with the /b (both on the backup and on the comparison test) and come back to you.
    Thanks a lot for your answer

    - - - Updated - - -

    Try3 said:
    Windows' count of the in C:\Windows files while Windows is running & its estimation of their size are known to be useless.
    - That's why tools such as Treesize as recommended for people investigating issues within C:\Windows.

    So this thread seems to be about comparing the incorrect count of files with another count of files derived from that incorrect count of files.

    No correct answer can possibly come from this.

    Antonio - If your aim is to get a full copy of the C:\Windows folder then boot from an InstallUSB to do it. You can still use RoboCopy.
    - Size comparisons might still be rubbish. C:\Windows includes some items [such as many within the component store] that appear to be files but are really hard-wired redirections to files that are elsewhere within C:\Windows. I have no idea how RoboCopy will cope with them.
    - I think you'd be better off making a system image.

    Denis
    I tried with Treesize and it gives me exactly the same result as File Explorer. Then I compared source and backup with WinMerge and the result seems to indicate that neither File Explorer nor Winmerge are accessing some files and folders in the source directory.

    My initial question is partly solved by adding /B to the robocopy comparison command:
    robocopy F:\Windows E:\roboRubi\Windowsz /L /NJH /NJS /NP /NS /B /log:C:\compare.txt
    ...but I don't understand the output file, it seems to show all of the file transfers instead of only the differences? what I need is the differences.

    Backup and compare with robocopy not working-screenshot-1394-.png
    Backup and compare with robocopy not working-screenshot-1396-.png
    Backup and compare with robocopy not working-screenshot-1395-.png
      My Computer


  9. Posts : 2,773
    Windows 11 Pro, 21H2
       #9

    You forgot to add the /NDL switch I noted previously. Without it, robocopy will list every blasted directory whether there are any differences or not. Add the /NDL and only file differences will be shown. Note that the full path for each file will still be shown. Also, missing and extra directories will still be shown. The /NDL will merely suppress the listing of every directory that is not relevant to actual differences.
      My Computers


  10. Posts : 163
    Windows 10
    Thread Starter
       #10

    I'm still researching this:
    Regarding how File Explorer counts files and folders:

    File Explorer count of files and folders (right-click a folder > properties), What is counted and what is not:
    I did an extensive test with a folder that contained all kind of files and folders: file shortcuts, folder shortcuts, system files, hidden files, file hardlinks, file symlinks, offline files (Onedrive files), Junction Points and Directory Symlinks.

    FOLDERS: Junction Points and Directory Symlinks aren't counted. Folder shortcuts aren't counted (these are counted as files!). Hidden folders, system folders etc, all are included in the count.

    FILES: all types were included in the count: file shortcuts, FOLDER SHORTCUTS (YES, THESE ARE COUNTED AS FILES!), system files, hidden files, file hardlinks, file symlinks, offline files (Onedrive files). Again: FOLDER SHORTCUTS ARE COUNTED AS FILES.

    Stuff of any kind within folder shortcuts, within JPoints, and within folder symlinks IS NOT COUNTED as neither files nor folders, same with folder shortcuts within folder shortcuts: not counted.
    Backup and compare with robocopy not working-screenshot-1695-.png

    Windows 10 20H2
    Tested on NTFS folder.
      My Computer


 

  Related Discussions
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd
All times are GMT -5. The time now is 06:14.
Find Us




Windows 10 Forums