How can I dedicate more RAM to file transfer?

I'm in the process of copying large (25-100 GB) blocks of data from several HDDs to a 4TB HDD just purchased. The transfer begins fast then slows to a snails pace. I partially understand that maybe buffering getting to capacity may be a problem. But I look at the performance tab/resource monitor in Task Manager and see that RAM is not nearly being used at capacity. I've closed some programs that run in the background but didn't make much difference.

See the attached picture.

How can I increase performance? I'm assuming that I need to release more RAM to be used for the file transfer?? Maybe I need to do something else - any advice will be appreciated.

For large file transfers, I like to use Free File Sync:
FreeFileSync: Open Source File Synchronization Backup Software

I encounter something similar under certain conditions...

My laptop is modest, i5 and 4Gb RAM with SSD and when copying large files of that sort of size (20/30Gb) I consistently get speeds in the 100MB/s region.

If I copy lots of small items within larger stuff (and I see you have nearly 30,000 showing there) then I get transfer rates dropping through the floor, perhaps as low as a few 10's of kb while the numerous small stuff is copied. In my case the small stuff is all .txt files of very small size.

I have never looked into reasons why this slow down occurs and simply assumed it was a consequence of having to access, move and constantly update the file table on the receiving drive.

Hi there
Basically it's all about Buffering -- there's not much you can do if you have zillions of small files.
RAM isn't really the issue here but the Windows file system itself. On those 4TB passport size drives you should be able to achieve on USB3 port around 110 MB/s (note not Mb/s) for average size files.

If you are using an internal 4TB drive then the speed will depend on the Disk cache size and the RPM speed of drive --- the SATA interface won't be a limiting factor here. If you are using "El Cheapo" brands e.g 5400 RPM with small cache size then you are stuck !!!! but a decent modern 4TB drive should still give you better than the external USB3 drive - so speeds of up to 250 MB/s (again not 250 Mb/s) should be possible.

What a better solution might be is to back the HDD up with say Macrium - Image the HDD rather than clone it and then restore to your new drive. I know it sounds Bonkers but the I/O in Macrium is streets ahead of standard Windows File explorer.

Same issue though for internal HDD -- image source disk and restore to target will probably work faster in your case than bog standard file explorer copy.


Cheers
jimbo

NavyLCDR said:

For large file transfers, I like to use Free File Sync:
FreeFileSync: Open Source File Synchronization Backup Software

Or FastCopy or TeraCopy, which also bypass Explorer's buffer allocation algorithms and use their own.

The reason is basically that there is a very large number of small files in the mix. This always leads to very much slower speed on the small files.
It is always like that doing a simple file copy with Windows File Explorer. I presume it is partly the Drives and partly the file system that does that. RAM does not come into this.

You can change that by ZIPing in to one large file, copying, and unZIPing again.