NTFS - time for Ms to create new file system ?


  1. Posts : 11,247
    Windows / Linux : Arch Linux
       #1

    NTFS - time for Ms to create new file system ?


    Hi there

    I posted recently about how slow NTFS was when you were copying large amounts of data - especially if the data consisted essentially of zillions of small files.

    Here's a reliable test which amply demonstrates that it really is time for Ms to at least consider re-writing the NTFS file system - especially as more and more of even HOME users have increasing amounts of data. The problem isn't per se in the maximum capacity of the file system but the way the I/O is performed.

    Here's a brief image

    NTFS - time for Ms to create new file system ?-snapshot11.png

    For those interested the article is here

    Linux 4.0 Hard Drive Comparison With EXT4 / Btrfs / XFS / NTFS / NILFS2 / ReiserFS - Phoronix

    Cheers
    jimbo
      My Computer


  2. Posts : 5,439
    Windows 11 Home
       #2

    I wonder, if it has something to do with linux itself, according to other test, 10 performs better than previous Windows.

    Flexense - Data Management Software - Windows 10 Disk Performance Review
      My Computer


  3. Posts : 11,247
    Windows / Linux : Arch Linux
    Thread Starter
       #3

    TairikuOkami said:
    I wonder, if it has something to do with linux itself, according to other test, 10 performs better than previous Windows.

    Flexense - Data Management Software - Windows 10 Disk Performance Review
    Hi there
    that could be true - however most people seem to think that NTFS isn't a particularly efficient file system anyway. Ms were working on a newer system --they use some type of new file system when you use Pooled volumes (use a group of volumes as a Single storage space). That could be an avenue for Ms to persue.

    Cheers
    jimbo
      My Computer


  4. Posts : 3,257
    Windows 10 Pro
       #4

    You never let me down Jimbo... Always first to demand that Microsoft do something they have already done.

    ReFS - Wikipedia, the free encyclopedia
      My Computer


  5. Posts : 2,832
    Windows 10 Pro X64
       #5

    Hi,

    ReFS is still not implemented in consumer systems as far as I know.

    NTFS is slow because it emphasizes security and data integrity.
    Any way you turn it, it all comes down on what you want to compromise really.

    Add a myriad of small files to be copied (meaning opening and closing every single one of them) and Windows wanting to check every file viruses and god knows what plus the fact that actions such as copying, moving and other file actions aren't high on the priority list, you inevitably end up with a long waiting time.
    A time depending on your CPU, chipset, whatever.

    All in all I find copying/moving file I W10 slow. I.e. way below what the hardware is capable of. it would be nice if we or W10 could set the priority of the file handling task to high when nothing else is running or demanding priority.

    Cheers,
      My Computers


  6. Posts : 7,254
    Windows 10 Pro 64-bit
       #6

    There was talk of Windows getting ReFS years ago but nothings happened.
      My Computers


  7. Posts : 3,257
    Windows 10 Pro
       #7

    swarfega said:
    There was talk of Windows getting ReFS years ago but nothings happened.
    ReFS is built into Windows 8.x and up. It's used by StorageSpaces for instance, but can also be enabled for normal drive usage, though it cannot be a boot volume.

    How You Can Try the New Resilient File System for Windows

    Having said that, ReFS will probably take 10 years before it's ready to replace NTFS, just like NTFS took a number of years before it replaced FAT(32).

    ReFS does not perform well (yet) on every condition, and in at least one case (sequential writes) it can be very slow. But I imagine that will change over time.

    SQL Server and ReFS: Part 2 FIO Benchmarking NTFS vs. ReFS | Exchange Spill

    I was really just commenting that Microsoft was already doing what Jimbo was suggesting. It doesn't mean it's ready to be used as a primary filesystem yet, but it's being done.
      My Computer


  8. Posts : 11,247
    Windows / Linux : Arch Linux
    Thread Starter
       #8

    Mystere said:
    ReFS is built into Windows 8.x and up. It's used by StorageSpaces for instance, but can also be enabled for normal drive usage, though it cannot be a boot volume.

    How You Can Try the New Resilient File System for Windows

    Having said that, ReFS will probably take 10 years before it's ready to replace NTFS, just like NTFS took a number of years before it replaced FAT(32).

    ReFS does not perform well (yet) on every condition, and in at least one case (sequential writes) it can be very slow. But I imagine that will change over time.

    SQL Server and ReFS: Part 2 FIO Benchmarking NTFS vs. ReFS | Exchange Spill

    I was really just commenting that Microsoft was already doing what Jimbo was suggesting. It doesn't mean it's ready to be used as a primary filesystem yet, but it's being done.

    Hi there

    I'm willing to have a go with this on a test system -- note using this on a VM isn't actually a good test since VM itself will use its own I/O (from the HOST system) to perform Disk transfers. (Assuming that they use something like VBOX or VMWARE VM software).

    I'll see if I can test this out on a REAL machine with a whole slew of small files (the most difficult for efficient and fast data transfer).

    As to File / Data Integrity --I haven't ever lost data with Linux EXT3/EXT4 or XFS - journalling system usually keeps file system pretty safe -- of course if Hardware goes defective that's another issue). I haven't lost data either with NTFS or even FAT32 so I don't think data integrity of the basic file system is a problem.

    A good thing with Linux file systems though is you don't need things like GPT or MBR -- the GPT system always creates a small "Microsoft reserved partition " - when you mount a GPT disk in Linux it always shows for example as /dev/sdd and /dev/sdd1 where /dev/sdd1 is the data partition and /dev/sdd shows up as something like a 500 MB "Microsoft reserved basic partition". With things like EXT4 / XFS you can create a single partition file system (mkfs.xfs /dev/sdd -f) which creates a file system on the entire HDD no matter what the size is.

    Of course Ms need to support legacy file systems but it's time now to get really efficient file I/O as HDD's become HUGE - and faster. !! SSD's while great aren't going to replace spinners for large data capacities just yet --especially when for example you can get fast 5TB 7200 RPM HDD's with 128 MB Cache for around 110 EUR / 120 USD a pop. !!!

    Cheers
    jimbo
      My Computer


  9. Posts : 4
    Windows 10
       #9

    I've been using ReFS on my Windows 10 box for over 6 months now and have had no problems. I set it up using Storage Spaces, and it uses two 5TB WD Blacks as a mirror. My decision to use this was I want to run Windows software, but wanted ultimate redundancy. I've heard of files going bad silently due to bad blocks and such. ZFS is supposed to prevent this, but it is next to impossible to find Skylake motherboards that support ECC memory (a ZFS requirement). I can run a HD Tune on it if anyone wants the stats.
      My Computer


 

  Related Discussions
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd
All times are GMT -5. The time now is 23:24.
Find Us




Windows 10 Forums