Difference of folder's size vs size on disk by multiples

witefoda · 14 Mar 2021

One folder I checked the properties of showed 55GB of files but "Size on disk" was 300+GB... I understand how they can be different, but I'm unsure of what that much of a difference actually means.

Difference of folder's size vs size on disk by multiples-folder-55gb-size-319gb-size-disk.png

Is 300GB of drive space actually being used?

It seems like a fragmentation issue, though I know there's a ton of small files (like code package dependency files) which could be making that huge difference, but I'm kind of wondering if this could cause any sort of roadblock to saving data or performance issue, and maybe I should zip up some folders.

ignatzatsonic · 14 Mar 2021

I've can't recall seeing a discrepancy that large.

I've always attributed the discrepancy to slack space and/or fragmentation.

Have you taken a gander at TreeSize or WinDirStat?

skcusime · 14 Mar 2021

Each file takes a minimum of 1 cluster (typically 4K) even if the file is just 1 byte. You have over a million files! So it's mathematically possible for the Size On Disc to be 4000 times bigger than the size of the files.

Berton · 14 Mar 2021

Also, the cluster size is fixed for all files but if a file is larger than that it will need two or more clusters and there will nearly always be the last bit of a file in its own cluster regardless of the size of the data, leaves a portion of that last cluster empty. A cluster can't contain any portion of data from a different file which could/does cause corruption and possible file lost. A large number of small files will take more space than what the total indicates because more clusters are needed, creates a supposed issue. One way to prevent that is to put all the small files in a single compressed file which then is what is stored, can help when the available storage space is limited and any 'wasted space' will exist only in the last cluster of those holding that file as all the other involved clusters will be full.

ignatzatsonic · 14 Mar 2021

skcusime said:

Each file takes a minimum of 1 cluster (typically 4K) even if the file is just 1 byte. You have over a million files! So it's mathematically possible for the Size On Disc to be 4000 times bigger than the size of the files.

Exactly so.

I just checked my data drive:

Only 130,000 files...of all types, ranging from 4k to 23 GB.

Size: 930 GB

Size on disk: 930 GB

Identical.

Your extreme amount of smallish files causes the distortion.

Would likely be different if your cluster size were adjusted.

Berton · 14 Mar 2021

ignatzatsonic said:

Would likely be different if your cluster size were adjusted.

A caution on that, the drive or partition on it would have to be reformatted to make that change so any data would be lost.

kreemoweet · 14 Mar 2021

It is not true that a file must take a minimum of 4K space on disk. Very small files, on a NTFS file system, can have all
their actual data stored in the Master File Table, in the 1K file record for that file. Even if it were true, a million or so files
could only have a maximum of 4G slack space, so would not even come close to the discrepancy the OP has.

Now, if the OP had lots of files with large amounts of data in alternate data streams, that might explain the situation.
As I recall, the size given for a file typically ignores any except the main data stream, but nevertheless the file would require
large amounts of allocated space on disk to store it.

Bree · 14 Mar 2021

For those unfamiliar with alternate data streams, see https://docs.microsoft.com/en-us/sys...nloads/streams

And if you think you don't have any files with ADS, then think again. Any file you downloaded that says 'This file came from another computer...' has that data stored in its ADS.

Unblock File in Windows 10