What should be the Cluster Size of 18 TB external hard drive

The average wasted space from Clusters is roughly. Cluster size /2 * Files Stored.
Meaning the bigger the cluster the more space you waste over stored files.

Its between two factors optimal storage space and speed.

if you opt for storage space you want the default allocation size or a smaller size.
if you want theoretically better speed then go for a higher cluster size.

The speed increase from a larger cluster would net trivial gains if at all. Hard drives a quite fast today to the point i think making a bigger cluster size is not really positive you won't really notice the difference its going to be microscopic levels of improvement if any.

The main benefit of larger cluster would be that the hard drive does not have to access as many cluster to retrieve the data and this in theory reduces wear and tear on the hard drive.
The other area it would benefit would be storing large files on the the drive to where you are not concerned about space and will never be reaching the drives capacity.

In the end i think default cluster size is the best and overall its set and forget regardless of what you do with the drive anything outside of this you have to be mindful of what you do with it

Thanks to this thread, I just realized one of my newer 14 TB Seagate external drives was pre-formatted as ExFAT(!) instead of the normal NTFS they've been doing for years. Silly me, the default block size is a whopping 256K.

Now I have find another large drive to transfer my files before reformattng. Would have been easier had I noticed this mistake before filling up the drive to 75% capacity...

https://techcommunity.microsoft.com/...fs/ba-p/425960

x509 said:

If you go with the default cluster size, you might avoid issues with third-party disk utilities that have the defaults baked in.

Reliable 3rd party disk utilities that can support NTFS should be properly aware that all versions of NTFS can support 4, 8, 16, 32 and 64 KB cluster size (up to 2 MB cluster size is supported by NTFS on Windows 10 v1709 / Windows Server 2019 and later). NTFS file compression is only available on NTFS volumes with 4 KB cluster size. But the reason why, on an 18 TB disk, the default cluster size is 8 KB instead of the more commonly found default cluster size of 4KB is because using a cluster size of 4KB, the maximum NTFS volume size is only 16 TB minus 4 KB so there would be a significant waste then. For extremely small files, maybe it should also be noted that NTFS doesn't allocate any clusters, as instead it stores the entire file content directly in the MFT (Master File Table) if the combined size of the file data and metadata attributes are small enough to be able to fit inside a single MFT entry, which is 1024 bytes.

Storage is so ridiculously cheap these days that it pays to overspend and go with the potential waste due to default cluster size, compared with the value of your time dealing with strange outcomes from third-party utilities.

Choosing to use the default cluster size instead of a larger cluster size is what reduces (not increases) the waste of storage space that results from storing a very large number of relatively small files. That's why I like to choose a cluster size bigger than the default on all my external HDDs, it helps to reduce the rate at which fragmentation gradually builds up from using the disk, and it takes such an annoying long time to defrag an external USB 3.0 large capacity HDD. (IIRC, Diskeeper 14 was the last version of Diskeeper that was still capable to do its thing on USB disks... which it did rather nicely IMO. lol)

garlin said:

Thanks to this thread, I just realized one of my newer 14 TB Seagate external drives was pre-formatted as ExFAT(!) instead of the normal NTFS they've been doing for years. Silly me, the default block size is a whopping 256K.

Now I have find another large drive to transfer my files before reformattng. Would have been easier had I noticed this mistake before filling up the drive to 75% capacity...

Glad that you found my thread useful :)

- - - Updated - - -

hdmi said:

Reliable 3rd party disk utilities that can support NTFS should be properly aware that all versions of NTFS can support 4, 8, 16, 32 and 64 KB cluster size (up to 2 MB cluster size is supported by NTFS on Windows 10 v1709 / Windows Server 2019 and later). NTFS file compression is only available on NTFS volumes with 4 KB cluster size. But the reason why, on an 18 TB disk, the default cluster size is 8 KB instead of the more commonly found default cluster size of 4KB is because using a cluster size of 4KB, the maximum NTFS volume size is only 16 TB minus 4 KB so there would be a significant waste then. For extremely small files, maybe it should also be noted that NTFS doesn't allocate any clusters, as instead it stores the entire file content directly in the MFT (Master File Table) if the combined size of the file data and metadata attributes are small enough to be able to fit inside a single MFT entry, which is 1024 bytes.

Choosing to use the default cluster size instead of a larger cluster size is what reduces (not increases) the waste of storage space that results from storing a very large number of relatively small files. That's why I like to choose a cluster size bigger than the default on all my external HDDs, it helps to reduce the rate at which fragmentation gradually builds up from using the disk, and it takes such an annoying long time to defrag an external USB 3.0 large capacity HDD. (IIRC, Diskeeper 14 was the last version of Diskeeper that was still capable to do its thing on USB disks... which it did rather nicely IMO. lol)

So what software do you think is the best to use for defragmenting big 16 or 18 TB hard drives. I also used to use diskeeper back in the day but don't know if its still around and kicking with new version or not.

kufr said:

Glad that you found my thread useful :)

- - - Updated - - -



So what software do you think is the best to use for defragmenting big 16 or 18 TB hard drives. I also used to use diskeeper back in the day but don't know if its still around and kicking with new version or not.

Diskeeper is these days called DymaxIO, but USB attached storage is no longer supported by this software like I said (eSATA─external SATA─should still work AFAIK). To reduce the time it takes to defrag via USB you might want to look into the kind of free defraggers that let you choose what files you want to be defragged.

WinContig
Download Defraggler for free | Defrag SSD and HDD drives
UltraDefrag download | SourceForge.net

Not sure if DiskTrix is worth paying for, but it could be.
DiskTrix Inc > Home