Extremely slow file transfer from computer to external HDD
I am almost sure it has to do with Malwarebytes real time protection which I recently installed, and I am not a complainer but this is almost ridiculous.
Robby
Unfortunately transferring 1000's of small files takes longer than 1 large file, Your better off just compressing them all together with Zip or 7zip and then transfer them.
Is the drive a USB 2 drive, If it's a USB 3 drive is it plugged into a USB 3 port.
The external HDD are connected through USB, but it's a somewhat older computer with only USB 2 ports.
And yes there are a lot of small files which I need to back-up frequently.
Robby
AV etc often causes it to crawl as it scans on read and on write
Most likely, that's the downside of real time protection
I noticed a huge speed difference in copying from NTFS to NTFS is 30 x faster
Sadly my 2 (4Tb HDD) are formatted in exFat, and I can not change it anymore.
Robby
If your 4TB disks are formatted ExFAT then each sector is 128kb. With NTFS it is (by default) 4kb. See this link:default-cluster-size-for-ntfs-fat-and-exfat
Apart from ExFAT wasting space for small files, what is interesting is the difference in cluster size (32x bigger) is exactly the same as the performance penalty you noticed. Perhaps it is a co-incidence - it would need more testing I guess.
Not sure what (if anything) can be done about it though if you can't reformat.
One question though - I see your example was photos from 2017. Are you backing up everything each time or just changed files? If you are repeatedly backing up the whole directory you could use robocopy instead and only write changed objects to backup disk.
The two external HDD are identical with files, so I started the journey to format the first one to NTFS and copy all the files from the other one (15 hours estimated)
Robby
You could also use SyncToy for backing up files and folders. Once the initial back up completed, it'll only back up changes. SyncToy can be scheduled to synchronize the folders and files in the background.
Download SyncToy 2.1 from Official Microsoft Download Center
I have mine running in every four hours, without noticing much of a performance hit on the system.
Quote