Compressing USB 3.0 external hard drive taking forever on high end PC

Ornithophilia · 18 Jul 2020

I have a 1TB external USB 3.0 hard drive plugged into a USB 3.0 port that is about half full ≈500 GB full. I decided to compress the drive so I am able to fit more on it since it is already half full. I decided to use my HP Envy 17 j177nr because it has an 8 core processor, 16 GB ram, and an Nvidia GeForce 840M so I figured that this computer could do the job much faster than my desktop that has only a duo core.

However, it has been more than 12 hours since I started and it is not even half way done! It says that there are still 14 hours to go!!! This PC is taking its sweet time compressing the drive.

My duo core desktop compressed nearly 2 TB of data on a USB 2.0 external hard drive in a USB 2.0 port in only a few hours. This laptop has four times as many cores and four times as much ram and is taking a significantly long amount of time to do much smaller job than the desktop did with the 2 TB drive last year! To add insult to injury, A clean install of Windows 10 was done on this laptop last Wednesday!

Does anybody know what the problem could be and how to speed this up?

If a low end desktop can compress 2 TB of data using USB 2.0 in only a few hours, there is NO REASON a PC with these specs should be taking more than 24 hours to compress 500 GB of data, 1/4 the amount of data that the low end PC did in only a few hours

thomaseg1 · 18 Jul 2020

Speed of compression often depends on type of files and number.
Pdf tend to be the slowest since they are already compressed somewhat.
Zip, 7z, Rar and even some exe files are compressed too. this will slow the process down
as it is trying to compress something that is already compressed.
Corrupted files will slow things down.
the sheer amount of files as well. 500g can mean 1000's of files.

Personally if the files aren't in much use, 7zip set to maximum compression
is probably a better way to go. set it to say 50gig zips and name them so you know whats in them.

mngerhold · 19 Jul 2020

I would echo thomaseg1's comments: I just did some tests, and compresing large, already compressed files was slow, but I think in my case limited by the disk write speed: A 4.5GB file took 3.5min on an old HDD (~20MB/s) but still took 2 min on a fastish NVME drive (40MB/s). The latter is much slower than normal disk transfers (200MB/s). I did notice that the compress routine only used one core, and on the HDD (at 20MB/s) it was under 50% used, and on the SSD was maybe 90%. So having 8 cores is no help! It would be worth comparing the single core performance of your two machines.

BTW, 500GB taking 24h is ~5MB/s (that is slow), whilst doing 2TB (if it was full, of course) in a few (say 3) hours is ~150MB/s, faster than you are going to get with USB2.

All that said, with only half the drive used, I wouldn't bother compressing - and if I did, I'd take the big files that might compress, and use 7zip. It is also worth noting that finding any large file that is not already compressed is hard - i wondered if my speeds would improve with a more compressble file, and had to resort to an AMD driver install folder to find anything that would compress (mostly .DLLs) - it was no quicker. I even ripped a WAV file from a CD to give me something I thought would be compressible, and the W10 compress option reduced it from 257 to 256 MB! 7zip did manage to reduce it to 204MB (default options), taking 30s. Quite what the W10 option was doing for the 10s it was busy I have no idea.