Windows 10: slow access to hd with millions of files

  1.    22 Nov 2017 #1

    slow access to hd with millions of files


    We have our security cameras recording to a 3 TB hard drive and it tends to get loaded with LOTS of files, well into the millions, loaded into subfolders based on camera, year, month, whether photos or videos (ex: f:\hikvision2\2017\01\photos). Access to this hard drive has become cumbersome - it takes minutes just to get a directory listing.

    I've already gone down the "change to general file type instead of photos" route, still slow access.

    I'd be interested in getting input on how to best address this issue and be able to maintain a history of files from the security cameras.
      My ComputerSystem Spec

  2.    22 Nov 2017 #2

    Possibly switching to a NAS RAID server or USB 3.0 RAID server with at least dual hard drives? If you went all out for a 4 drive servers, there are different raid modes that would offer both drive failure recovery capability and spanning across more than one drive to increase performance.

    USB 3.0 raid enclosure - Newegg.com
      My ComputerSystem Spec

  3.    22 Nov 2017 #3

    Windows Explorer has always had problems with large numbers of files/folder. You can run Process Monitor - Windows Sysinternals | Microsoft Docs to see what files Explorer is accessing just to confirm it's not doing anything weird. But from personal experience once you get into the 10s of thousands of files there's not much you can do.

    You can try a different file explorer app or do things from the command line. There may be existing solutions out there for security systems like this. But a custom front-end wouldn't be hard to write if the folders are well-organized.

    Edit: This is assuming you have thousands of files/folders within a single folder level (e.g. \folder\00001, \folder\00002, ... , \folder\99999).
      My ComputerSystem Spec

  4.    22 Nov 2017 #4

    The hard drive is actually in a Synology NAS, so, don't know if that's got anything to do with the access performance. But, there are thousands of jpg files in each directory - when a camera triggers the system sends jpg snapshots to the hard drive, at least 5 pics each time it triggers. At night when we have spider webs waving across the camera, we get triggers about each 10 sec, all night long. Same when fog rolls across all the cameras: boom a lot of cameras trigger and send snapshots do the hard drive.

    So, the hard drive is chock full of jpg files and even getting to the drive to delete files is a pita since you have to navigate down the directory tree and each move takes several minutes.
      My ComputerSystem Spec

  5. Farvatten's Avatar
    Posts : 36
    Windows 10 Pro 64bit 1809 17763.1
       22 Nov 2017 #5

    Use Date Modified instead of Date in the Column Headers


    Not sure if this helps you but I've seen my response times improve when I did this:

    Slow Folder Loading when sorting by Date Solved - Page 2 - Windows 10 Forums
      My ComputersSystem Spec

  6.    23 Nov 2017 #6

    You said that you have tried "general file type instead of photos" without success.
    I, too, was unsuccessful when choosing "photos".

    Instead I find that the best choice is "documents".
    Try that. It worked for me with both Windows 10 and Windows 7.
      My ComputerSystem Spec

  7.    23 Nov 2017 #7

    No change, I've tried that route several times with no change.
      My ComputerSystem Spec


 

Related Threads
I have a big problem. I have a NAS with some folders containing many movies, in one 1.000 and in the other over 2.000, and every time I go to open the folder windows 10 takes a long to see them all files, especially put them in order of date or size...
Disc access very slow in Performance & Maintenance
I have an HP Envy i7 laptop which came with a 1TB hard drive. As there was space to add a second drive I added a 500GB drive. In addition I added a 320GB drive in the DVD slot (using a dvd/disc converter cartridge). I have recently noticed that...
Hello. I have an older PC (Phenom 1, crappy nForce chipset) that was running slow under Win7. I have installed Win10 from scratch (x64) and it's running better but when trying to access some shared folders with another Win10 computer, it takes a...
Clean install of w10. I am signed in w/a local account which is also an admin account on the PC I'm saving/moving too. I have read/write access to these folders but I cannot save to the folders. Any idea what's going on? 33597
Windows 10 disk access seems slow in Performance & Maintenance
I have noticed on a clean install of Windows 10 that disk access is sometimes slower than 8.1.1 by quite a bit. I have a 960 gig Crucial SSD that used to be blazing fast. Sometimes when I click on drove C windows does a search to see what is on it....
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd
All times are GMT -5. The time now is 04:45.
Find Us