Best way to Clean-up a huge File Structure keeping only the good.

Page 1 of 2 12 LastLast
  1.    15 Dec 2018 #1

    Best way to Clean-up a huge File Structure keeping only the good.


    Doing this manually is nearly impossible. The folder structure contains over 160,000 files that have accumulated over many years. Less than half of these need to be kept. But they are buried among all the other files that need to be removed.

    I think it might be possible using some vartiation of robocopy to move only the specified filetypes from folder A to folder B and create the entire path as it moved the files. I don't know what combination of switches would be needed to accomplish this though and it is important that the original folder structure be maintained.

    After I have moved as many types as i can think of, I could examine what was left in folder A to see if I missed anything. If anyone knows of a utility or tool made to do this I would appreciate the name
      My ComputerSystem Spec

  2. AndreTen's Avatar
    Posts : 16,020
    Windows 10 (Pro and Insider Pro)
       15 Dec 2018 #2

    You could use wiztree. It shows extensions too and about zillion other info. I prefer treesize for simple treemap, but Wiztree offers more info. There are some nice pictures on their site.

    Antibody - Wiztree
      My ComputersSystem Spec

  3.    4 Weeks Ago #3

    questorfla said: View Post
    Doing this manually is nearly impossible. The folder structure contains over 160,000 files that have accumulated over many years. Less than half of these need to be kept. But they are buried among all the other files that need to be removed.

    I think it might be possible using some vartiation of robocopy to move only the specified filetypes from folder A to folder B and create the entire path as it moved the files. I don't know what combination of switches would be needed to accomplish this though and it is important that the original folder structure be maintained.

    After I have moved as many types as i can think of, I could examine what was left in folder A to see if I missed anything. If anyone knows of a utility or tool made to do this I would appreciate the name
    Please provide an example and we'll create the robocopy command/commands.
    PowerShell can also be very helpful in filtering and copying file types.
      My ComputerSystem Spec


  4. Posts : 1,136
    Windows 10 Home x64 and Pro x86
       4 Weeks Ago #4

    Please define your criteria for "need to be kept".

    Please confirm that you have a file backup.

    Denis
      My ComputerSystem Spec

  5. Bree's Avatar
    Posts : 9,937
    10 Home x64 (1809) (10 Pro on 2nd pc)
       4 Weeks Ago #5

    questorfla said: View Post
    I think it might be possible using some vartiation of robocopy to move only the specified filetypes from folder A to folder B and create the entire path as it moved the files...
    ddelo said: View Post
    Please provide an example and we'll create the robocopy command/commands.

    I think I'd use the older Xcopy for this (in fact, I have) as RoboCopy needs a few more switches to cope with unexpected things like junction points or locked files. It's defaults can be less than useful in such circumstances. I wrote myself a 'user files backup' batch file with RoboCopy and until I got the switches right it could hang or go into an infinite loop.


    Both RoboCopy and Xcopy have a /S option that will copy subdirectories. For Xcopy a simple command to copy all .mp4 files from folder A and recreate the folder structure in Folder B would be:

    XCOPY /S "Folder A\*.mp4" "Folder B\"
      My ComputersSystem Spec

  6.    4 Weeks Ago #6

    Several have asked for an example. I made this as complete as I could along with all the things that i need to accomplish when run. (Please excuse typos i tried to catch them alI)
    It should MOVE only a single file type such as *.doc to the same path in B2 as it existed in A1. Nothing else would get moved or changed in A1. All the *.doc files in every subfolder in A1 would be moved to their same path in B2.

    B2 is currently an empty copy of the full directory tree of A1 and I only need to populate it with the desired files in their correct places. However, if the script could create the subfolder paths as needed when it runs that would save me time on future uses. That would also avoid a lot of empty subfolders in B2 that I am sure to end up with and will have to remove anyway.

    Subsequent runs of the same script should be able to next move *.txt or any other file extension to and from the same folders without duplicating any previously created paths by using the paths where they exist in B2. No duplicate or (2) subfolders created.

    When job is complete, A1 would have its complete original folder tree lacking only the files that have been moved to their same locations in B2.
    Here is the simplest possible layout showing what i am trying to do

    A1\
    item.zip
    other.doc

    sub1\
    pic.doc
    file.ini

    sub2\
    text.doc
    text.exe
    text.cfg

    sub3\
    temp.fil
    _______________________________________________
    B2\
    sub1\
    sub2\
    sub3\

    First run would move only all *.doc files from C:\A1\ to C:\B2 in the same subdirectory paths as they previously existed in A1. Shown below is C:\B2 after the *.doc move

    C:\B2\
    other.doc

    sub1\
    pic.doc

    sub2\
    text.doc

    sub3\

    This needs to be run as a batch script and would be Super if it asked which file type to move as well as the name of the original directory and destination directory and used them as a variables in the script.
    Example:

    Set org = "enter folder to move from: " (this needs to require full path as c:\a\b\c for example
    Set dst = "enter folder to move to: " (this also needs to require full path as c:\a\b\c )
    Set ty = "Enter file type to move: "
    Then use %org% and %dst% with %ty% in the script.
    (this is to allow the script to be in the same folder i keep all my scripts in but be run at a prompt anywhere)

    At the end of the first run it should loop back to ask for "file type" again and run through the same move
    And keep this up until the File Type I enter is "stop"
    This has to be done on several folders for different reasons and if it works as needed I would want something I could save and easily reuse for future needs.

    PS: Here is the script that got me the Empty dir-tree copied from A1 to B2 but I cannot find the right switches to get it populated as described above. It does work perfect though and if i could get the right switches in it in the right places I believe it could do it all.
    Code:
    robocopy.exe /R:0 /W:0 "C:\A1" "C:\B2" /E /Z /Copy:DT /XF "*.doc" /LOG:"C:\SCRIPTS\EMPTY.LOG"
    Last edited by questorfla; 4 Weeks Ago at 13:55.
      My ComputerSystem Spec

  7.    4 Weeks Ago #7

    Thanks BREE!!
    I got so focused on trying to force Robocopy to work that i totally missed your Simple Xcopy routine.
    It Worked perfectly (I think). And a batch file using it wil be easy to modify with the needed variables. I can't believe it was that easy but sometimes the hardest things are. Tunnel-vision is a terrible thing to get!
    The Only part that Xcopy didn't do was remove the original files after copying, which is what i need to happen to get things as I need them. The starting folder should have all the chosen filestypes Moved to the new folder and kept in their correct paths. After copying, the files themselves, (not the folders they were in) need to be removed from the original so it ends up with only the original directory tree which would then include only filetypes that were not moved out of it.
    I am hoping there is a "switch for that" in Xcopy and i'm looking right now.
    Or, as an altrnative, since i already have a way to get an empty directory tree, maybe the MOVE command can be used to get the files into it? But i have never gotten out of "move" what I expected to get.
      My ComputerSystem Spec

  8. Bree's Avatar
    Posts : 9,937
    10 Home x64 (1809) (10 Pro on 2nd pc)
       4 Weeks Ago #8

    questorfla said: View Post
    This needs to be run as a batch script and would be Super if it asked which file type to move as well as the name of the original directory and destination directory and used them as a variables in the script.
    Example:
    Set org = "enter folder to move from: " (this needs to require full path as c:\a\b\c for example
    Set dst = "enter folder to move to: " (this also needs to require full path as c:\a\b\c )
    Set ty = "Enter file type to move: "
    Then use %org% and %dst% with %ty% in the script.
    (this is to allow the script to be in the same folder i keep all my scripts in but be run at a prompt anywhere)
    Couple of point there, the SET command as used above requires the /P option. Also, you need to perform some sanity checks on the answers provided. eg.

    Code:
    :ORG
    SET /P org="enter folder to move from: "
    if not exist %org% echo the %org% folder does not exist
    if not exist %org% goto ORG
    echo .
    echo Folder "%org%" selected.
    For the file type you'll need to check if it was entered with or without a leading 'dot', ie was doc or .doc typed?

    Code:
    :TY
    SET /P ty="Enter file type to move: "
    SET _1st=%ty:~0,1%
    if "%_1st%" == "." goto TY-OK
    echo "please enter a file type with a leading '.' eg: .txt"
     goto TY
    :TY-OK
    echo.
    echo file extension %ty% selected.
    reference: variable substring - Windows CMD - SS64.com
      My ComputersSystem Spec

  9. Bree's Avatar
    Posts : 9,937
    10 Home x64 (1809) (10 Pro on 2nd pc)
       4 Weeks Ago #9

    questorfla said: View Post
    The Only part that Xcopy didn't do was remove the original files after copying, which is what i need to happen to get things as I need them. ... I am hoping there is a "switch for that" in Xcopy and i'm looking right now.
    Or, as an altrnative, since i already have a way to get an empty directory tree, maybe the MOVE command can be used to get the files into it? But i have never gotten out of "move" what I expected to get.
    No, neither Xcopy or Move have the options you want.

    I'm always wary of deleting files from within a batch command as there's no recycle bin 'safety net' to fall back on*. However, if you want to delete the original files after a successful copy, then the DEL command has a /S option too.


    * this came up earlier, powershell or 3rd party utilities can make the recycle bin a viable option
    Command Prompt to delete to recycle bin - Windows 10 Forums
      My ComputersSystem Spec

  10.    4 Weeks Ago #10

    Thanks for catching all my dumb errors The set missing /p etc. is what i get for trying to post before even test/running. While I agree 100% with the dangers of using delete in batch, the worst that could happen is i would just make another copy of the original and try again. I think i will go with your del /s After the rest is done though

    I never run any kind of experimental scripts on original folders anyway. When finally completed i would be running it on a copy of the folder i need to clean. But I am always one of the 'fools to rush in where angels fear to tread' and always appreciate the 'heads-up'.
    Like i said, it is the EASY ways that I seem to have complete overlooked. While examing Xcopy option i found i could have much easier used it with /T /E to create my Empty Directory tree as well. Again Thanks.

    One final: I decided to enter the original path to be including file-type to avoid the other problem with missing (.) etc. Since it will only be Me running it i will remember much of the needs but you are 100% correct. I should consider my own limitations and possible errors if doing this for a "keep around" script.
      My ComputerSystem Spec


 
Page 1 of 2 12 LastLast

Related Threads
Old Dell inspiron Was running Windows 7, never had BSOD. Clean installed Windows 10 Home 64bit, only has avast and used portable spybot a couple of times. Drivers seem OK on device manager. bluescreenviewer shows caused by ntoskrnl.exe ...
the tutorial here says to delete all partitions before clean installing windows 10. But I don't want to do that. I just want to reinstall cleanly on the OS partition only. In the past with win7, when I had an OS and Data partition, I would...
Hi, I have to install Windows 10 Pro in a bootable hard drive of about 1Tb (model WD10EARS), formatted like this: first an 850Gb NTFS partition with data, and then 150Gb of unallocated space. Currently no operating system is installed on the disk....
Hi everyone, I've been having problems with my computer BSOD'ing, even under light usage (internet browsing, etc). The computer will crash randomly, sometimes several times an hour, or sometimes it will go for a week with no problem. It started a...
Hey guys, I know this is an opinionated question (as it is with many software questions). I just upgraded to Windows 10 (I did a SecureErase on my SSD with Parted Magic followed by a complete clean install). I would like to keep my PC as clean...

Tags for this Thread

Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd
All times are GMT -5. The time now is 02:49.
Find Us