How can I dedicate more RAM to file transfer?

Page 1 of 2 12 LastLast

  1. Posts : 304
    Windows 10 home Version 10.0.19041 Build 19041
       #1

    How can I dedicate more RAM to file transfer?


    I'm in the process of copying large (25-100 GB) blocks of data from several HDDs to a 4TB HDD just purchased. The transfer begins fast then slows to a snails pace. I partially understand that maybe buffering getting to capacity may be a problem. But I look at the performance tab/resource monitor in Task Manager and see that RAM is not nearly being used at capacity. I've closed some programs that run in the background but didn't make much difference.

    See the attached picture.

    How can I increase performance? I'm assuming that I need to release more RAM to be used for the file transfer?? Maybe I need to do something else - any advice will be appreciated.

    How can I dedicate more RAM to file transfer?-ram-usage.jpg
      My Computers

  2. Porthos's Avatar
    Posts : 881
    Win 10
       #2

    I usually turn off my AV when I am copying large amounts of data from drive to drive I don't need to be scanning the data during transfer. Of course, don't do anything else at the same time.
      My Computers

  3. NavyLCDR's Avatar
    Posts : 15,515
    Windows 10 Pro
       #3

    For large file transfers, I like to use Free File Sync:
    FreeFileSync: Open Source File Synchronization Backup Software
      My Computer


  4. Posts : 1,868
    W10 pro x64 20H2 Build 19042.610
       #4

    I encounter something similar under certain conditions...

    My laptop is modest, i5 and 4Gb RAM with SSD and when copying large files of that sort of size (20/30Gb) I consistently get speeds in the 100MB/s region.

    If I copy lots of small items within larger stuff (and I see you have nearly 30,000 showing there) then I get transfer rates dropping through the floor, perhaps as low as a few 10's of kb while the numerous small stuff is copied. In my case the small stuff is all .txt files of very small size.

    I have never looked into reasons why this slow down occurs and simply assumed it was a consequence of having to access, move and constantly update the file table on the receiving drive.
      My Computer

  5. Infrasonic's Avatar
    Posts : 245
    W10 Home 1903 (Build 18362.295)
       #5

    A software RAM drive might be able to help, I've seen them successfully used for speeding up file transfer, but never done it myself.
    You'll still run into the HDD cache issue once it maxes out as the amount of data you want to transfer will exceed any RAM drive limits too, it might just lengthen the initial phase speed up though. Dual channel RAM might help there also, but that's a guess.
      My Computers

  6. jimbo45's Avatar
    Posts : 10,484
    Windows / Linux : Arch Linux
       #6

    Hi there
    Basically it's all about Buffering -- there's not much you can do if you have zillions of small files.
    RAM isn't really the issue here but the Windows file system itself. On those 4TB passport size drives you should be able to achieve on USB3 port around 110 MB/s (note not Mb/s) for average size files.

    If you are using an internal 4TB drive then the speed will depend on the Disk cache size and the RPM speed of drive --- the SATA interface won't be a limiting factor here. If you are using "El Cheapo" brands e.g 5400 RPM with small cache size then you are stuck !!!! but a decent modern 4TB drive should still give you better than the external USB3 drive - so speeds of up to 250 MB/s (again not 250 Mb/s) should be possible.

    What a better solution might be is to back the HDD up with say Macrium - Image the HDD rather than clone it and then restore to your new drive. I know it sounds Bonkers but the I/O in Macrium is streets ahead of standard Windows File explorer.

    Same issue though for internal HDD -- image source disk and restore to target will probably work faster in your case than bog standard file explorer copy.


    Cheers
    jimbo
      My Computer


  7. Posts : 191
    Windows XP, 10; Knoppix [Debian] linux
       #7

    NavyLCDR said:
    For large file transfers, I like to use Free File Sync:
    FreeFileSync: Open Source File Synchronization Backup Software
    Or FastCopy or TeraCopy, which also bypass Explorer's buffer allocation algorithms and use their own.
      My Computer

  8. f14tomcat's Avatar
    Posts : 53,157
    Multi-boot Windows 10 - RTM, RP, Beta, and Insider
       #8

    mike s said:
    Or FastCopy or TeraCopy, which also bypass Explorer's buffer allocation algorithms and use their own.
    TeraCopy also has option to verify copy, not sure about FastCopy.
      My Computers

  9. Infrasonic's Avatar
    Posts : 245
    W10 Home 1903 (Build 18362.295)
       #9

    It's a couple of years old but there's a speed comparison here...15 Free File Copy Tools Tested for the Fastest Transfer Speeds • Raymond.CC
      My Computers


  10. Posts : 1,963
    Windows 10
       #10

    The reason is basically that there is a very large number of small files in the mix. This always leads to very much slower speed on the small files.
    It is always like that doing a simple file copy with Windows File Explorer. I presume it is partly the Drives and partly the file system that does that. RAM does not come into this.

    You can change that by ZIPing in to one large file, copying, and unZIPing again.
      My Computer


 
Page 1 of 2 12 LastLast

  Related Discussions
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd
All times are GMT -5. The time now is 19:14.
Find Us




Windows 10 Forums