Windows 10: Win10 NTFS file copy/backup utility that handles permissions correctly

1. Win10 User, If you're not going to use a professional tool like Paragon, then both Bree & "slicendice" have given you some great options for backing up directories and files. Sorry, we can't offer much info or thoughts on SyncToy, save to say it is Microsoft, and it might be a great option for you; however we don't use any Sync features at all, and disable the ability to sync data on our systems in general for security.

With xCopy - Be sure to use it with the following switches in Command Line to keep the Permissions, Ownerships, Auditing Information, etc.

EXAMPLE xCopy in Command Prompt:

xcopy /x /k /b /e /g /f c:\xdrivedata\*.* x:\drivedata

Be careful with the " /g " switch because that will allow copying of encrypted data to a destination that doesn't support encryption, and will cause the encryption on those same files to be lost at the point they're copied to. The original files will keep their encryption. Sometimes it is necessary to use the " /g " switch, but other times not. When and when not to use the " /g " switch is strictly at your discretion.

With RoboCopy, just be mindful of the following from the Wiki Bree was kind enough to post the most, because there may be cases when you want to use xCopy or a professional tool that allows for more precision control:

[BEGINS QUOTE FROM WIKI];
Folder copier, not file copier

Robocopy syntax is markedly different from standard copy commands, as it accepts only folder names as its source and destination arguments. File names and wild-card characters (such as "*.*") are not valid source or destination arguments.
However, files may be selected or excluded using the optional "file" filtering argument (which may include wildcards) along with various other options.[14] File specifications can refer only to the filenames relative to the folders already selected for copying. Fully qualified path names are not supported.
For example, to copy the file foo.txt from directory c:\bar to c:\baz, one could use the following syntax:
robocopy c:\bar c:\baz foo.txt
And to copy only PDF files — with file extension .pdf — from directory c:\bar to c:\baz, one could use the following syntax:
robocopy c:\bar c:\baz *.pdf
;[ENDS QUOTE FROM WIKI]

A lot of great ideas and preventative maintenance leads here, and definitely plenty of options to choose from.

Best of Luck,

SLI - IT-IS Team
My Computer

2. @Win10 User

I understand that Windows NTFS file system and application execution permissions can be a bit overwhelming.

When administering file system permissions the basic ideology is to restrict permissions as much as possible on the whole system as a baseline. And then only enable specific users to access specific folders.

When it comes to applications and services, one have to use the same ideology. For a user who tries to setup IIS server the proper way for the first time would be overwhelmed by the amount of settings that has to be configured before it works properly and securely.

Once one understand the basic ideology behind a NT system, it gets a lot easier.

The Administrator has always access to everything. But I am not talking about the Administrator group, but the built in Administrator (similar to root in Linux) that is disabled by default. One should only revert to this account during emergencies or when setting up things that only this account has access to. After that it should be secured behind a very complicated password and disabled again.

There is a whole science behind this whole NT ideology and it requires a deeper education in order to understand the whole thinking process.
My Computers

3. IIS = "aye-ayeeeeeeeee-esssssssssssssssssss!" LOL

" @slicendice " , just wondering, have you tested things with the root Administrator account lately? If you've got the latest updates, you may very well find yourself unpleasantly surprised.

As far as privileges and permissions go, it's actually been our own experience to keep things simple for the Administrators group in that the following are to be done:

1 - We Enable the Internal (ROOT) Admin Account, and also set it so that it cannot be used remotely.

2 - We setup a Second Admin Account, throw in several layers of security that include our own TLD and CA servers with RADIUS and Directory Services via x.500 with Windows 10 in a Standalone WorkStation Configuration, that is a WorkGroup Member so that both Machines and Users have to compartmentally ID to be authorized on our Private Cloud. This approach has worked for well over 10 years without incident.

3 - Obviously the Second Admin account is the most used by Machine Administrators, so assigning rights specifically to either the Administrators Group or the Accounts directly has worked fine in the past without any problems, and performance on the latest builds is still the same from our own testing and observations.

4 - The ShadowCopy service can still work in our configuration, but we don't really consider that to be a "Syncing" service because ShadowCopy as a service was originally intended solely for Data Backup to be allowed while Windows is in live operation, and even though we've got it locked down very heavily on both software and even our hardware firewalls, ShadowCopy will still run as we need it to, but in no other cases such as "Syncing" will ShadowCopy operate.

NOTE ON ITEMS 2, 3, and 4: The implementation we use is supported by proxying Internet Access; essentially making us our own ISP while optimizing portability for authorized equipment. This method ensures Machines are Registered to our infrastructure, and for primary support of the infrastructure, we utilize Novell Linux Enterprise Server (a.k.a. NLES). A purely OpenSource variant of NLES is of course OpenSUSE that can work equally as well, we just prefer NLES because it's a bit more concrete for us and easier to register to our Enterprise.

SLI - IT-IS Team
My Computer

4. There are lots of reasons for tree-copying/tree-syncing the entire file system, such as cloning the c: drive so that one can compare the "saved" system stuff against the current running stuff, or having a second set-up to use for "experiments" (such as a DVD-based re-install). "sfc /scannow" is supposed to be able to restore system files, but it's broken on my Win10 (wasn't so for previous Win versions).

There are two issues in doing verbatim tree-copying. If the source tree is a "moving" target that can't be reliably snapshot in a consistent state, Windows has traditionally used volume shadow copy to make a duplicate of the source as necessary. If the copying mechanism has to over-write the destination tree (as in an incremental sync copy), then it can't be locked out of modifying the destination tree (as it is now the case with Win10). For example, there is a "really hidden" directory at c:\Windows\Winapps that's owned by TrustedInstaller. If a "first" copy (by an utility running under Administrator) of c:\Windows\Winapps is made to f:\Windows\Winapps, then the copy/sync will succeed. However, some subsequent copy of some stuff in c:\Windows\Winapps to f:\Windows\Winapps could likely fail because the destination Winapps is owned by TrustedInstaller, and can't be modified by Administrator.

Linux doesn't have this problem, as root can overwrite anything. Windows obviously instituted TrustedInstaller to plug some security hole where some app that manages to run under Administrator wreaks havoc on some system files. Win10's explorer apparently uses some new "smart copy" mechanism to deal with some of the legitimate needs to copy/modify/delete system files, but it's not clear smart copy works with command-line tools.

Xcopy/shadowcopy/robocopy probably can't deal with the destination-tree permission lock problem, and can only do copy anyway (verses sync, where only modified files are copied). Syncing is often much more preferred over dumb copying, since there may be 500MB file system out of which only 20MB needs to be updated at the destination. Linux's rsync utility does this very nicely (can even re-start the sync properly, in case of interruptions).

With these inherent "design flaws," it's not clear to me that doing a Win10 re-install with _not_ saving user files and apps would do me any good, as the underlying problem will continue to exist, and there is a good chance my apps may not be able to re-install properly.

A related, hysterical, problem I'm having is that Windows update is being forced upon us. Since my Windows Update is broken, I see my eventvwr repeatedly download and try installing update packages. While many are shown as succeeding (which I'm suspicious about, given that these packages keep getting updated), the big bad one is KB4088776, which is the critical cumulative update for 1709, which keeps failing to install. To add insult upon injury, Microsoft decided that even if an update fails, if it required a restart after installation (whether succeeding or failing), the system is restarted. So, my machine gets rebooted almost once a day.
Brilliant. If some company is so arrogant to force things on its customers, then it better get the thing right. Prior to Win10, one can mark an update for no re-try (although the cumulative update should really be installed)..
My Computer

5. Our issue with Syncing is that in some cases Syncing can be exploited, and then just hooking up a high capacity drive and using the sync services can cause a serious data breach. Eliminating use of Syncing is the right choice for us, but we certainly recognize its not the right choice for everyone.

The issues @Win10 User has brought up, definitely require a little R&D work. A community effort would certainly be helpful in figuring out the best approaches.

SLI - IT-IS Team
My Computer

IIS = "aye-ayeeeeeeeee-esssssssssssssssssss!" LOL

" @slicendice " , just wondering, have you tested things with the root Administrator account lately? If you've got the latest updates, you may very well find yourself unpleasantly surprised.
Actually I haven't. I will experiment with this for the next couple of days/weeks and see what works and what not.

Shadow Service should still work as intended and thus being able to copy the whole running system. I could be wrong though. Only testing will prove one or the other.
My Computers

7. @slicendice - We've noticed Microsoft's slipped a few past the goalie using updates in the recent past, but as long as things are regularly checked, it's pretty easy to retrace and re-implement any settings that become lost. Existing permissions can be helpful in preserving settings as well, so if you've hit any golden combinations, please let us all know.

Looking forward to hearing more.

Our Best Always,

SLI - IT-IS Team

"Let us proclaim the mysteries of IT!"
My Computer

• The saying "there is no security where physical isolation/security is missing" always rings true. If some hacker/insider hooks up a USB drive and boots off some live-DVD, then the (then off-line) system (or critical data) drive could easily be cloned. Locking down sync tools on an on-line system just makes it a little more difficult to breach critical data (when the system has to be left on-line), but the physical media where any sync is destined still needs to be physically retrieved.

In a lab environment, I've used Linux's rsync to easily clone one Unix system to another, as that's often a desired mode of operation for doing experiments and code development.

The Win10 server I'm currently trying to fix is my home system. I don't have any of the concerns with multi-user use, insider hacking, etc., although I have to deal with hardware failure, power failure, etc., that could corrupt things.

I really just want to get my Windows Update working correctly again. The fact that "sfc /scannow," "dism," "Windows troubleshooting," and a miriad of other Win10-designated ways of fixing the system don't work is a manifestation of how Microsoft dropped the ball on this critical aspect. It doesn't help to have all these fancy schemes to plug up security holes when the code to implement these schemes can't even be applied (as the case of the cumulative update not installing).

Using the big-hammer approach of a full re-install is not a viable answer for many home users (or tiny businesses) as we have lots of old apps that work fine currently (but may not be re-installable). In a large work environment, the IT department would deal with these things, and have funds/manpower to re-install from scratch.
My Computer

• Win10 User said:
I really just want to get my Windows Update working correctly again. The fact that "sfc /scannow," "dism," "Windows troubleshooting," and a miriad of other Win10-designated ways of fixing the system don't work is a manifestation of how Microsoft dropped the ball on this critical aspect. It doesn't help to have all these fancy schemes to plug up security holes when the code to implement these schemes can't even be applied (as the case of the cumulative update not installing).
OK!

Let's start with this WU CU issue. What have you done in order to resolve this issue, except for running sfc, dism and system check?

Have you tried to disable fast startup, temporarily disable AV and finally, temporarily stopped all WU related services and deleted everything in C:\Windows\SoftwareDistribution\Download folder and rebooted system at least 2 times (first time will end up in WU error message if it was already trying to install something)?

What about Hypervisor support? Have you tried booting without hypervisor support? I've solved many issues in the past by doing this in Admin CMD:
Code:
bcdedit /set hypervisorlaunchtype off
shutdown /r /t 0

To re-enable hypervisor type:
Code:
bcdedit /set hypervisorlaunchtype on
OR
Code:
bcdedit /set hypervisorlaunchtype auto
My Computers

• @Win10 User , You're completely right about that, and all too often Microsoft goes script crazy and just recites "Run SFC /Scannow" & "dism.exe /online /cleanup-image /restorehealth" to a degree that it's almost a cultesque mantra now. Unfortunately it's a mantra that's been catching on in the industry, and keeps people from doing any real troubleshooting because they're constantly having to reset their equipment followed by addressing even more issues related to any corruption that occurs after the reset.

Are you using Group Policy at all to manage Windows Updates?

The reason we ask is because we often retest a very focused configuration for Windows Updates within Group Policy where everything is setup accordingly in a very specific manner for each setting; however for the past 2 years, it's never worked properly despite the settings being spot on. We do use a minimal configuration to prevent Windows Update from meddling with drivers, but obviously in our configuration, we're just using the straight up Windows Update, and we don't support use of WSUS because we use a Linux-based backend for everything else.

If you think it would help, it'll take us some time, but we could post the Windows Updates settings that we do keep configured, and then just bear in mind that all other settings for Windows Updates in our Machine Group Policy are set to non-configured. This approach has kept Windows Updates working for us without any meddling from Microsoft's proprietary and often very mediocre drivers that don't work nearly as well as the hardware component's manufacturers drivers often do. The only exception we've ever found to this is actually using things like Memory Card Reader-Writer assemblies, but that's about all Microsoft Drivers have been good for in our own experience.

We're all too aware, you've probably already done this, but instead of our assuming, we have to ask just to cover it; Have you tried re-reregistering services, repairing the Visual C Runtimes, and repairing the Microsoft .NET runtimes?

Please let us know if you think any of our offerings would be helpful.

Best Always,

SLI - IT-IS Team

"IT personnel using scripts are like actors, 'Oh, I'm not really a technician, but I play one at work.'"
My Computer

•
Page 2 of 5 First 1234 ... Last

Hi folks If you use Linux on a NAS or on a VM (Virtual Machine) there's an alternative to the Windows Mp3tag (very good but unfortunately suffers from problem windows has with long file names -- not Mp3tag's fault I hasten to add). Program is...
How to Backup and Restore Permissions of Files, Folders, or Drives in Windows On NTFS and ReFS volumes, you can set security permissions on files and folders. These permissions grant or deny access to the files and folders. Every container...
Hello, I'm have an issuing when copying a file from a Windows 10 workstation to a FQDN Windows server 2003 resource. The environment is as follows; SERVER Windows 2003 SP2 Server (x86) (Domain Controller) FQDN = server1.company.local IP...
NTFS Permissions Confusion in General Support
I clearly do not understand NTFS permissions. Can someone please help eliminate my confusion. When I am signed in to RAVEN\Clayton (local, standard account) and I check Properties->Security on D:\Users\Clayton\Utilities, I find: 30157 To me...
Hi there I'm sure I haven't added any extra software - but Windows 10 TO GO version seems to handle READ and WRITE to a LINUX EXT3/4 File system. I know Linux can handle Windows file systems but this is the first time where I've seen WINDOWS...
Our Sites