Page 3 of 3 FirstFirst 123
  1.    15 Jun 2017 #21
    Join Date : May 2017
    Posts : 76
    Windows 10 Home, 64bit
    Thread Starter

    You choose Best Performance to enable write caching, which means you will/should use the Safely Remove feature before removing the drive.You choose Quick Removal when you do not want to use the Safely Remove feature.
    Yes, but why did you ask: Curious if you have tried setting the removal policy on Best Performance?
      My ComputerSystem Spec
  2.    15 Jun 2017 #22
    Join Date : Aug 2015
    Posts : 611
    W10 Home 1703

    Because it wasn't clear (to me) whether that had been done.
      My ComputerSystem Spec
  3.    15 Jun 2017 #23
    Join Date : May 2017
    Posts : 76
    Windows 10 Home, 64bit
    Thread Starter

    Alright, I understand.

    Just had the same probleme with another drive, it couldn't be removed safely.


    There obviously were two recycle bins on it. After some tries / error messages

    I made it to remove the bins.



    Suddenly I could remove that drive safely.
      My ComputerSystem Spec
  4.    15 Jun 2017 #24
    Join Date : Aug 2015
    Posts : 611
    W10 Home 1703

    Very cool.
    Hope it works for the other drive too.
      My ComputerSystem Spec
  5.    15 Jun 2017 #25
    Join Date : May 2017
    Posts : 76
    Windows 10 Home, 64bit
    Thread Starter

    Yes, very strange, so obviously the bins indeed have a connection to these error messages (how should one know that), so your idea to mention that was absolutely great.

    Thank you very much
      My ComputerSystem Spec
  6.    16 Jun 2017 #26
    Join Date : Mar 2015
    Posts : 15
    Win7

    @Enter and @TV2: Sorry, I'm also not set up for email notification. Very interesting discussion since my last post.

    A few additional thoughts/questions ...

    Quote Originally Posted by Enter View Post
    FreeCommander is closed but there are still shown some P: processes in Process Monitor:


    How could I delete / close them?
    Hmmm... I don't know and I can't figure out how to replicate or create something similar on my system, I suspect because I don't have the 'Operation' column in Process Explorer. I've never seen that column before, and I can't figure out how to activate it on my Win7 system. How did you get it to display? (Is it perhaps only available in Win10?)

    Do those FreeCommander P:\ line entries have any associated handles if you open handles in the lower pane (Ctrl+H)? When you see this kind of entry after FreeCommander is closed, does a Process Explorer search (Ctrl+F) for 'P:\' find any open handles associated with FreeCommander? If not, then it doesn't seem like deleting/closing the P:\ line entries would be necessary before removing the external USB drive.

    Quote Originally Posted by Enter View Post
    In Process Explorer this is shown:


    Looks somehow risky to close these handles.
    I haven't previously noticed handles like the AIMP.exe handles shown here (without the drive P\: in the handle Name, and with 'Section' as the handle Type). Do those handles close if you close AIMP?

    At any rate, I'm guessing they are not relevant to the inability to safely remove the external USB drive, as it appears you searched for "P:" rather than "P:\", and those handles were found because they include "P:" in the handle Name (AIMP:MouseHookShare and AIMP:KeyboardHookShare).

    Quote Originally Posted by enter View Post
    And what if windows is just accessing the drive / ntfs system or so?
    Quote Originally Posted by enter View Post
    But what if windows scans / rebuilds or whatever that ntfs?
    Without researching it further (which I don't have time or energy to do right now), I'm still not 100% certain what produces the ...$Extend\$RmMetadata\$Txf handles. I previously mentioned from recollection of my research a few years ago that these handles are associated with NTFS journaling, but if I remember correctly, commenters in some forums/blogs thought they might be associated with Windows Indexing or Recycle Bin. My best recollection is that after sifting through everything I could find, I was about 90-95% certain the culprit was/is NTFS journaling.

    I've since forgotten much of what I learned at that time, but this MSDN blog post by Raymond Chen (Jan 1, 2013) seems to explain the relevant basics of NTFS journaling in regard to removing an external USB drive. If I understand it correctly, the journaling is the first of several steps whenever changes are made to a file on an NTFS drive, and "provides file system recoverability by logging ... operations required for any transaction that alters important file system data structures ... before these operations are carried through on the disk" (e.g., before all other steps necessary to complete copying/altering/deleting a file). File data loss/corruption problems can occur if an external USB drive is removed before all the other steps are completed. As you obviously already understand, with write caching enabled, it is possible that the remaining steps may not actually be completed when the OS informs the user that the copying is complete, as the file changes cached in RAM may not yet written to the removable drive (presumably depending on how busy the OS is, data transfer speed, etc.). So their could be data corruption if the drive is unplugged before those steps are completed.

    But disabling write caching is supposed to avoid such problems, so file operations are supposed to be complete when the OS tells you they are complete:
    It seems to me that if journaling is the first step in any file-system change, then once you can be confident the other steps involved in file changes are completed, obviously no further journaling is necessary and the drive can be removed. Basically, as I understand it, journaling protects the file system structures (preventing file system corruption), while disabling write-caching protects the data contents inside those already protected file system structures (preventing file data corruption due to premature unplugging).

    So my reasoning is that *IF*, in fact, the ...$Extend\$RmMetadata\$Txf handles are created/used by NTFS journaling as I have surmised, then "yanking the drive" with those journaling handles still open shouldn't cause any problems, as long as you verify other handles are closed and you have allowed enough time for all other file operations to complete. (Usually should be a few seconds at most, but pretty cautious and I figure I'm safe if I've waited 10-15 minutes.)

    ...But this reasoning hinges on whether I'm correct about the source of the ...$Extend\$RmMetadata\$Txf handles.

    Given the ubiquity of external USB drives and the frequency of the ...$Extend\$RmMetadata\$Txf handles problem, it has always seemed strange to me that Microsoft has not (AFAIK) at a minimum:

    • authoritatively clarified the issue, so users can be confident about what is going on (Is it NTFS journaling?)
    • provided some guidance on 'safe' practices or workarounds (Is it 'safe' to unplug if those are the only open handles?)


    ...But I don't think those minimums are actually sufficient (especially since most average users would never see them or probably understand them. Instead, if these ...$Extend\$RmMetadata\$Txf handles are, in fact, associated with journaling, it seems to me Microsoft should build into Windows some way of automatically closing the handles when the user attempts to "Safely Remove Hardware and Eject Media", or provide a GUI option to do so, if/when that is the only issue preventing safe removal.

    Quote Originally Posted by Enter View Post
    Just had the same probleme with another drive, it couldn't be removed safely.
    This reminds me that when I previously researched this, there was also some discussion in the Everything Search forum about whether NTFS indexing by Everything Search might be preventing external USB drive removal (and maybe something about whether Everything made use of the ...$Extend\$RmMetadata\$Txf handles ???). I don't recall the issues clearly right now, but I think there were some suggestions about how to tweak the Everything NTFS Index settings that seemed to make sense. ...But IIRC correctly I tried the suggested remedies without success.

    Quote Originally Posted by Enter View Post
    There obviously were two recycle bins on it. After some tries / error messages

    I can see that this refers to what appears to be a Recycle Bin (S-1-5-21-263......), but any chance you could post an English translation of the first 5 lines?

    Quote Originally Posted by Enter View Post
    I made it to remove the bins.


    Suddenly I could remove that drive safely.
    Ohhh, very interesting!!!

    Did you check what happened to the ...$Extend\$RmMetadata\$Txf handles in Process Explorer after you deleted the $Recycle.bin? ...I'm assuming they must have been closed/gone, but I'm trying to understand if deleting the $Recycle.bin closed the handles. If so, maybe they don't have to do with NTFS journaling, or perhaps NTFS journaling and Recycle Bin interact somehow (???)

    I'm glad you've been looking at this so closely and willing to put so much time into researching it and posting your results, as it's a very frequent and vexing problem, and it would be nice to get a clearer understanding. I put a couple questions in boldface and blue above to make them easier to pick out. Like I said earlier, I'm not set up for email notification, but I'll try to check back regularly for the next few days to continue our discussion if you have anything further to post.
      My ComputerSystem Spec
  7.    16 Jun 2017 #27
    Join Date : Jul 2015
    Posts : 3,752
    10 Pro

    Quote Originally Posted by gdv View Post
    I don't have the 'Operation' column in Process Explorer. I've never seen that column before, and I can't figure out how to activate it on my Win7 system.
    'Operation' is a default column in ProcMon (that is Process Monitor as in the screen shot - not Process Explorer). If you mean ProcMon and you don't have the 'Operation' column you can add it by going to Options > Select Columns.

    As for unplugging drives I have the same issue with the journal apparently locking volumes but I couldn't ever figure it out. It is more of a problem for me disconnecting the (SATA) disk in my docking station than USB but happens with both.

    I use a shortcut to sync.exe pinned to the taskbar to flush everything to disk and then just disconnect.
      My ComputerSystem Spec
  8.    16 Jun 2017 #28
    Join Date : May 2017
    Posts : 76
    Windows 10 Home, 64bit
    Thread Starter

    Hello gdv.

    Hmmm... I don't know and I can't figure out how to replicate or create something similar on my system, I suspect because I don't have the 'Operation' column in Process Explorer. I've never seen that column before, and I can't figure out how to activate it on my Win7 system. How did you get it to display? (Is it perhaps only available in Win10?)
    It is "Process Monitor", not "Process Explorer". May be "Process Monitor" is not that reliable, I do not know.

    Do those FreeCommander P:\ line entries have any associated handles if you open handles in the lower pane (Ctrl+H)? When you see this kind of entry after FreeCommander is closed, does a Process Explorer search (Ctrl+F) for 'P:\' find any open handles associated with FreeCommander?
    I could not reproduce that display at the moment, I assume since "Process Explorer" didn't show those handles there indeed were not such.

    If not, then it doesn't seem like deleting/closing the P:\ line entries would be necessary before removing the external USB drive.
    Yes, but forcing to close such a line would have the same effect, I assume, as just plugging off the drive, the risk of data loss / data loss.

    I haven't previously noticed handles like the AIMP.exe handles shown here (without the drive P\: in the handle Name, and with 'Section' as the handle Type). Do those handles close if you close AIMP?
    This now is "P Explorer". If I remember it right, yes, after Aimp has been closed these entries were gone.

    At any rate, I'm guessing they are not relevant to the inability to safely remove the external USB drive, as it appears you searched for "P:" rather than "P:", and those handles were found because they include "P:" in the handle Name (AIMP:MouseHookShare and AIMP:KeyboardHookShare).
    Yes, I would assume that, too.

    My best recollection is that after sifting through everything I could find, I was about 90-95% certain the culprit was/is NTFS journaling.
    Yes, those meta data are caused / used by the NTFS, I would say.

    File data loss/corruption problems can occur if an external USB drive is removed before all the other steps are completed.
    Yes, or if a handle is close by force, I guess.

    it is possible that the remaining steps may not actually be completed when the OS informs the user that the copying is complete, as the file changes cached in RAM may not yet written to the removable drive (presumably depending on how busy the OS is, data transfer speed, etc.).
    Yes, and / or besides of process belonging to that special transfer, there might be other processes, e.g. rebuilding / changing the NTFS, the Journal or whatever.

    So their could be data corruption if the drive is unplugged before those steps are completed.
    Yes, that's why one has to click "safely remove" to write the content of the cache to the drive. Actually I would assume, flushing the cache would have the same effect, but the error message keeps on popping up.

    It seems to me that if journaling is the first step in any file-system change, then once you can be confident the other steps involved in file changes are completed, obviously no further journaling is necessary and the drive can be removed. Basically, as I understand it, journaling protects the file system structures (preventing file system corruption), while disabling write-caching protects the data contents inside those already protected file system structures (preventing file data corruption due to premature unplugging).
    Yes, sounds plausible.

    So my reasoning is that *IF*, in fact, the ...$Extend\$RmMetadata\$Txf handles are created/used by NTFS journaling as I have surmised, then "yanking the drive" with those journaling handles still open shouldn't cause any problems
    Yes, sounds like a reasonable assumption, of course, but if not...one never knows, not only that a single file might get corrupt, if the NTFS breaks...

    So my reasoning is that *IF*, in fact, the ...$Extend\$RmMetadata\$Txf handles are created/used by NTFS journaling as I have surmised, then "yanking the drive" with those journaling handles still open shouldn't cause any problems, as long as you verify other handles are closed and you have allowed enough time for all other file operations to complete. (Usually should be a few seconds at most, but pretty cautious and I figure I'm safe if I've waited 10-15 minutes.)
    Or watching if the LED of the drive is still blinking, I could imagine.

    authoritatively clarified the issue, so users can be confident about what is going on (Is it NTFS journaling?)
    provided some guidance on 'safe' practices or workarounds (Is it 'safe' to unplug if those are the only open handles?)
    Or rather do some programming (or what ever) to solve the problem.

    This reminds me that when I previously researched this, there was also some discussion in the Everything Search forum about whether NTFS indexing by Everything Search might be preventing external USB drive removal (and maybe something about whether Everything made use of the ...$Extend\$RmMetadata\$Txf handles ???). I don't recall the issues clearly right now, but I think there were some suggestions about how to tweak the Everything NTFS Index settings that seemed to make sense. ...But IIRC correctly I tried the suggested remedies without success.
    Ooops, the best search program I know, may be it was me asking that. Everything would not cause failing savely removing, it would automatically close or release such handles to let the drive be safely removed.

    I can see that this refers to what appears to be a Recycle Bin (S-1-5-21-263......), but any chance you could post an English translation of the first 5 lines?
    Yes, of course:

    Access to folder denied.
    Admin rights needed to process.
    You have to get the rights from admin to make changes to the folder.

    Date of creation.
    Button, left: Try again and "Cancel", right.

    Did you check what happened to the ...$Extend\$RmMetadata\$Txf handles in Process Explorer after you deleted the $Recycle.bin?
    If I remember it right, they disappeared.

    .I'm assuming they must have been closed/gone, but I'm trying to understand if deleting the $Recycle.bin closed the handles. If so, maybe they don't have to do with NTFS journaling, or perhaps NTFS journaling and Recycle Bin interact somehow (???)
    I would assume, the Journaling / NTFS works the same way for bins like it / them does with normal files.

    I'm not set up for email notification, but I'll try to check back regularly for the next few days to continue our discussion if you have anything further to post.
    Yes, that's great, thank you.

    I use a shortcut to sync.exe pinned to the taskbar to flush everything to disk and then just disconnect.
    That one I use, too. But it is just flushing the cache, it does not effect / stop the journaling, NTFS actions, I would say. This one can be used, too: FFB.zip
      My ComputerSystem Spec
  9.    17 Jun 2017 #29
    Join Date : Mar 2015
    Posts : 15
    Win7

    Sorry to be so long getting back. This took a long time to write (too long), and I ended up doing some research during the writing. Then when I was about 2/3 done, the forum software somehow lost my reply content in the edit box during a preview, so I had to rewrite a lot of it.

    There almost certainly errors and typos, but I'm too wasted to proof it any more carefully right now.

    Quote Originally Posted by lx07 View Post
    'Operation' is a default column in ProcMon (that is Process Monitor as in the screen shot - not Process Explorer).
    I knew (and almost said in previous post) the answer was going to be obvious! ... .... ...But thanks!

    Part of my problem was that the image doesn't display full size in my browser, and I was having a hard time reading the relevant P: lines in question. By the time I downloaded the image from imgur.com so I could display it full size to read the P: lines, I had already given up trying to find the column in Process Explorer. ...So I didn't even look at the top left corner where I can now clearly see "Process Monitor". Worse, I'm already familiar with Process Monitor; but to be fair, I rarely use it, or the 'Time of Day' column (which I wondered about but didn't search for), the registry keys, etc. in the image would have tipped me off.

    Quote Originally Posted by lx07 View Post
    I use a shortcut to sync.exe pinned to the taskbar to flush everything to disk and then just disconnect.
    Ahhh, thanks! That seems like a good additional precaution. Although I always just make sure I've waited more than long enough before unplugging, I will incorporate that into my practices. Should especially be relevant if/when there is a need to unplug in a hurry.

    Quote Originally Posted by Enter View Post
    ...May be "Process Monitor" is not that reliable, I do not know.
    To the contrary, although I rarely use it, I think it is very powerful and reliable... ...but just not the right tool for the job of finding open drive handles.

    Quote Originally Posted by gdv View Post
    If not, then it doesn't seem like deleting/closing the P:\ line entries would be necessary before removing the external USB drive.
    Quote Originally Posted by Enter View Post
    Yes, but forcing to close such a line would have the same effect, I assume, as just plugging off the drive, the risk of data loss / data loss.
    When I thought the screenshot was from Process Explorer I was saying those P:\ line entries would not need to be closed before removing drive (because probably have nothing to do with handles on the P:\ drive). Now that I realize the screenshot is from Process Monitor , I would point out that the P:\ line entries cannot be forced closed.

    Process Monitor displays a running list (log) of the filtered process events that occur on the system between when you start and stop monitoring, so each of those lines simply represents a single process event that occurred at the time listed in the 'Time of Day' column.

    I don't know enough about it, but it seems likely to me that there are process events when a P:\ drive handle is opened or closed (and that might even be part of what the CreateFile and CloseFile P:\ line entries in the screenshot indicate ???). But if so, the resulting line entry in Process Monitor line simply shows when that event occurred. I guess if you knew the right kind of event to filter for, you could conceivably monitor for only handle open and close events, and then would be able to view when those occurred on the P:\ drive and check for a close following each open. But that would be an extremely inefficient way to observe handles, and you still wouldn't be able to force the handles closed from within Process Monitor.

    Quote Originally Posted by gdv View Post
    File data loss/corruption problems can occur if an external USB drive is removed before all the other steps are completed.
    Quote Originally Posted by Enter View Post
    Yes, or if a handle is close by force, I guess.
    Yes <- - - - - - - [Gee, I can almost never answer that briefly.]
    Quote Originally Posted by gdv View Post
    it is possible that the remaining steps may not actually be completed when the OS informs the user that the copying is complete, as the file changes cached in RAM may not yet written to the removable drive (presumably depending on how busy the OS is, data transfer speed, etc.).
    Quote Originally Posted by Enter View Post
    Yes, and / or besides of process belonging to that special transfer, there might be other processes, e.g. rebuilding / changing the NTFS, the Journal or whatever.
    Not completely sure I'm understanding you here, but if you have already verified in Process Explorer that all other handles are closed on the external drive, that particular transfer should be the only thing left to complete. The journaling crucial to NTFS drive recovery will have already occurred first, and there shouldn't be any other processes besides completing the operations of writing of any file changes to the drive (flushing any remaining data from the cache and writing it to the drive). No further journaling required, at least for drive recovery.

    Per NTFS Transaction Journal:
    NTFS log provides file system recoverability by logging, or recording, the operations required for any transaction that alters important file system data structures. This is done before these operations are carried through on the disk. This process ensures that if the system crashes, partially completed transactions can be redone or undone when the system comes back online

    As I understand it, the idea behind journaling is that any NTFS structural changes are logged to the drive first, before the file data changes (held in a cache) are actually written to/deleted from the drive. This protects the NTFS data structures so recovery to a stable state is possible in the event of system crashes, power failure, etc. (...including unplugging a USB NTFS drive). But obviously it doesn't protect the actual data to be stored in those structures if the data isn't yet written to the drive.

    So as I understand it, even if the file is quite large, the NTFS journaling includes logging of files names, logging where on the drive data is to be written/deleted, etc. Logging these events involves a trivial amount of metadata that can be written to the drive almost instantly/immediately from a human perspective, and is written before the writing of relevant data, so the NTFS logging events are not really a meaningful source of delays the user has to be concerned about. I could be wrong, but I'm assuming the OS won't represent to the user that the operation is completed before this crucial NTFS logging occurs. Then the remaining operations necessary to write the actual data to the drive occur (with each step in the process is also logged almost instantly in NTFS journal). What could take longer is flushing data from the cache and writing it to the drive, transferring large amounts of data over a limited USB transfer rate connection, etc. But unless another change is made to the file, no further NTFS logging is needed to protect drive file system recoverability, even if data is still being written (although still unwritten file data won't be recoverable).

    Again per NTFS Transaction Journal:
    After a transaction is logged, the NTFS performs the sub-operations on the volume itself, in the cache. Finally, after the cache has been updated, NTFS writes another record to the log file, recording that the transaction has been completed.

    When a system is being recovered after a system failure, the NTFS reads through the log file and redoes each committed transaction. After that, the NTFS locates all of the transactions in the log file that weren't committed at the failure and rolls back.

    As I understand it, after the data has been flushed form cache and written to disk, the final NTFS logging event occurs to mark the process as complete. Again this involves a trivial amount of metadata that can be written to the drive almost instantly. If the file data itself hasn't yet been completely written to the drive, the final NTFS log file record showing the file is complete won't be there, and NTFS rollback will occur upon recovery. But once the data writing is complete, the NTFS log entry should occur almost instantly, and the drive can then be safely removed. (I'm pretty sure, but not 100%, this also means that if a very large file is only partly written, most of the written data will be recoverable. Depending on the type of data it may or may not be usable: i.e., a plain text file, an .mp3 file, or some types of video files that can be split anywhere might work as expected up to the point where the transfer ended.)

    Quote Originally Posted by gdv View Post
    So there could be data corruption if the drive is unplugged before those steps are completed.
    Quote Originally Posted by Enter View Post
    Yes, that's why one has to click "safely remove" to write the content of the cache to the drive. Actually I would assume, flushing the cache would have the same effect, but the error message keeps on popping up.
    I assume probably the same re flushing the cache. But if the ...$Extend\$RmMetadata\$Txf handles exist for NTFS journaling (per our hypothesis) and are the only problem (and I think we can be confident they are once enough time has passed), the error message is popping up not because of uncompleted data writing, but because those handles are "locking" the drive. And the OS apparently does not have any means to automatically close them when no longer needed, and the user UI has no manual option to unlock/close them.

    Yes, sounds like a reasonable assumption, of course, but if not...one never knows, not only that a single file might get corrupt, if the NTFS breaks...
    See NTFS journaling info above. The NTFS logging is written almost instantly to the drive before, during, and immediately after the data is written, so there is no reason for NTFS to 'break'. At worst, MTFS will rollback to the last completed/logged transaction and the file system will be stable (not broken) going forward from there. At worst, any unwritten data in the one file that did not finish writing will be missing from that file, but the file metadata will be intact. Maybe an analogy would be a paper file folder (NTFS data structure) being placed in a file cabinet (HDD), but not all the relevant sheets of paper (data) get added to the file folder before someone closes the drawer and locks the file cabinet (system crash, power loss, unplug USB). When you unlock and open the file cabinet (reboot), the file folders are all intact, complete, and usable, except the one folder that was incomplete when the drawer got closed and locked. But even it's contents are in the proper folder and usable, although incomplete.

    If you allow enough time (or flush the cache), it seems even that last file written to the drive should be complete and sound. I fact after researching this a bit more and trying to write it up, I think I've convinced myself that I have been being way overly cautious in waiting at least 10-15 minutes (and often hours) before removing a drive. Unless I have been writing a hug file to the drive, if only the ...$Extend\$RmMetadata\$Txf handles exist, I think I've convinced myself that 1-2 minutes should be more than sufficient

    Or watching if the LED of the drive is still blinking, I could imagine.
    Might be helpful for those drives that have a blinking LED, but not all do (and most USB flash drives don't, but still could be formatted NTFS), and I don't know if they all behave the same way. I think I'd have more confidence in checking the handles in Process Explorer and adding in the additional step of flushing the cache with Sync or FFB (see below).
    Or rather do some programming (or what ever) to solve the problem.
    That's what I meant when I said,
    Quote Originally Posted by gdv View Post
    ...But I don't think those minimums are actually sufficient (especially since most average users would never see them or probably understand them. Instead, if these ...$Extend\$RmMetadata\$Txf handles are, in fact, associated with journaling, it seems to me Microsoft should build into Windows some way of automatically closing the handles when the user attempts to "Safely Remove Hardware and Eject Media", or provide a GUI option to do so, if/when that is the only issue preventing safe removal. (emphasis added)
    To make that a bit clearer, it seems like Microsoft should build in a way to detect when ...$Extend\$RmMetadata\$Txf handles are the only issue preventing drive removal, and then either automatically close the handles when the user clicks "Safely Remove Hardware..." or provide a UI checkbox or button for user to manually close those handles (maybe with brief description of the issue and guidance to help user decide whether to use the checkbox/button).
    Ooops, the best search program I know, may be it was me asking that.
    Yeah, I love Everything Search! I'll try to go back and check those Indexing discussions as soon as I get a chance. Do you use the same username there? (I'm gdv there as well)

    Everything would not cause failing savely removing, it would automatically close or release such handles to let the drive be safely removed.
    Are you saying that is the way it now works? ...or that settings can now be tweaked to do that? I haven't checked the site for a while, and I'm running v1.4.0.713b, which I know is not the most recent version. I'll try to check for updates as soon as I get a chance.

    Access to folder denied.
    Admin rights needed to process.
    You have to get the rights from admin to make changes to the folder.

    Date of creation.
    Button, left: Try again and "Cancel", right.
    Ahhh, thanks for the translation. Helps me better understand the whole $Recycle.bin issue.

    Quote Originally Posted by gdv View Post
    Did you check what happened to the ...$Extend\$RmMetadata\$Txf handles in Process Explorer after you deleted the $Recycle.bin?
    Quote Originally Posted by Enter View Post
    If I remember it right, they disappeared.
    If that is easy to replicate, or if you run into a similar situation, it would be nice to know for sure, as it might imply Recycle Bins are keeping the ...$Extend\$RmMetadata\$Txf handles open. Apparently it is established that a corrupted Recycle Bin is one cause for the handles not closing, I would then want to get some idea of how often that is the cause. What if it is the main cause, or even the only cause? I don't think that is the case as I don't think I'm likely to have corrupted Recycle Bins because I nearly always delete files completely (Shift+Delete) rather than sending them to the Recycle Bin

    Quote Originally Posted by gdv View Post
    I'm assuming they must have been closed/gone, but I'm trying to understand if deleting the $Recycle.bin closed the handles. If so, maybe they don't have to do with NTFS journaling, or perhaps NTFS journaling and Recycle Bin interact somehow (???)
    Quote Originally Posted by Enter View Post
    I would assume, the Journaling / NTFS works the same way for bins like it / them does with normal files.
    So would I, but if it is corrupted Recycle Bins that cause the handles to not be released, that is an important finding, and if deleting the corrupted Recycle Bin closes the handles (and results in them not staying open during future attempts to "Safely Remove Hardware...", that would be even more important.

    Quote Originally Posted by lx07 View Post
    I use a shortcut to sync.exe pinned to the taskbar to flush everything to disk and then just disconnect.
    Quote Originally Posted by Enter View Post
    That one I use, too. But it is just flushing the cache, it does not effect / stop the journaling, NTFS actions, I would say.
    Agreed, both Sync & FFB just flush the data in the cache to the drive, but theoretically that should stop any NTFS journaling as well, as that logging is done almost instantly before data is written, updated almost instantly while data is being written, and finalized almost instantly when the data-writing is completed (see discussion above). But what Sync and FFB don't do is close the NTFS ...$Extend\$RmMetadata\$Txf handles on the drive.

    To stay with the admittedly imperfect paper file folder analogy, the file folder (NTFS) is labeled (NTFS) and slotted into the correct section (NTFS) of the correct drawer (NTFS) of the filing cabinet (HDD), and all the relevant sheets of paper (data) are filed in the file folder, but the filing secretary (OS) is still holding onto the file folder (...$Extend\$RmMetadata\$Txf handles) inside the drawer, so the drawer cannot be closed and locked ("Safely Remove Hardware").

    Quote Originally Posted by Enter View Post
    This one can be used, too: FFB.zip
    Ahhh, I see you have also found USB expert, Uwe Sieber (although you probably use his German page). I don't think I've visited his site since I was researching the ...$Extend\$RmMetadata\$Txf handles a few years ago, and I don't think I grasped the issues well enough at the time to have realized FFB might be useful. (...Might have even been before FFB was first introduced in April 2013 but probably since then.) ...But I had been thinking while drafting each of my posts in this thread that I would go back and check his site for anything new on the issue as soon as I get a chance.

    According to Google, there is no mention of "$Extend\$RmMetadata\$Txf" (or "$Extend" or "$RmMetadata" or "$Txf") anywhere on his site, but he's the only one I've run across on the internet who I'm guessing could probably authoritatively clarify the ...$Extend\$RmMetadata\$Txf handles issue. I remember thinking about trying to email him about it a few years ago, but never got around to it.

    I notice he mentions a couple of limitations of Sysinternals Sync that FFB overcomes:
    It would be interesting to know if the FFB -f switch simply does the same thing as closing handles in Process Explorer:If so, the switch would presumably force close all open handles on the drive. That, along with detection of what is preventing 'Safely Remove Hardware", would be the kind of function I think Microsoft should build in to automatically use (or manually selectable by user), if Windows detects that only the four ...$Extend\$RmMetadata\$Txf handles are preventing "Safely Remove Hardware".
      My ComputerSystem Spec
  10.    17 Jun 2017 #30
    Join Date : May 2017
    Posts : 76
    Windows 10 Home, 64bit
    Thread Starter

    Sorry to be so long getting back. This took a long time to write (too long), and I ended up doing some research during the writing. Then when I was about 2/3 done, the forum software somehow lost my reply content in the edit box during a preview, so I had to rewrite a lot of it.
    Yes, that is really bad. Very sorry for that. But glad you are back.

    it is possible that the remaining steps may not actually be completed when the OS informs the user that the copying is complete, as the file changes cached in RAM may not yet written to the removable drive (presumably depending on how busy the OS is, data transfer speed, etc.).
    Enter said: View Post
    Yes, and / or besides of process belonging to that special transfer, there might be other processes, e.g. rebuilding / changing the NTFS, the Journal or whatever.
    Not completely sure I'm understanding you here, but if you have already verified in Process Explorer that all other handles are closed on the external drive, that particular transfer should be the only thing left to complete.
    Yes, that's right, I would say, if one watches the processes / handles - in this case - P: there should not be other ones, I would say, than the ones you see. But if one does not watch them - and usually it should be like that - one may be never knows, if there still is a process running, may be re-indexing the NTFS, rebuilding it or whatever.

    Per NTFS Transaction Journal:
    NTFS log provides file system recoverability by logging, or recording, the operations required for any transaction that alters important file system data structures. This is done before these operations are carried through on the disk. This process ensures that if the system crashes, partially completed transactions can be redone or undone when the system comes back online
    Yes, but sometimes it does not work, a recovery is not possible, so both the NTFS and the back up are corrupted obviously. May be the back up, recording, journaling work in another way (worse performance) when the option "Quick removal" is activated.

    As I understand it, the idea behind journaling is that any NTFS structural changes are logged to the drive first, before the file data changes (held in a cache) are actually written to/deleted from the drive. This protects the NTFS data structures so recovery to a stable state is possible in the event of system crashes, power failure, etc. (...including unplugging a USB NTFS drive). But obviously it doesn't protect the actual data to be stored in those structures if the data isn't yet written to the drive.
    Yes, and / or sometimes it obviously fails completely, the drive is shown as "raw".

    Yes, that's why one has to click "safely remove" to write the content of the cache to the drive. Actually I would assume, flushing the cache would have the same effect, but the error message keeps on popping up.
    I assume probably the same re flushing the cache. But if the ...$Extend\$RmMetadata\$Txf handles exist for NTFS journaling (per our hypothesis) and are the only problem (and I think we can be confident they are once enough time has passed), the error message is popping up not because of uncompleted data writing, but because those handles are "locking" the drive.
    Yes, that sounds plausible, may be one thought was, that, if the data in the cache is written to the drive Win updates the journal, NTFS and then there is no need anymore to keep the handle / process running.

    Yes, sounds like a reasonable assumption, of course, but if not...one never knows, not only that a single file might get corrupt, if the NTFS breaks...
    See NTFS journaling info above. The NTFS logging is written almost instantly to the drive before, during, and immediately after the data is written, so there is no reason for NTFS to 'break'. At worst, MTFS will rollback to the last completed/logged transaction and the file system will be stable (not broken) going forward from there. At worst, any unwritten data in the one file that did not finish writing will be missing from that file, but the file metadata will be intact.
    That is the reason why I do not move files generally but copy and after I delete the source files or use programs like FastCopy (doing it that way, removing the source files - when moving - after all of the files are transferred correctly)

    Yeah, I love Everything Search! I'll try to go back and check those Indexing discussions as soon as I get a chance. Do you use the same username there? (I'm gdv there as well)
    No, I do not, it is Biff.

    Everything would not cause failing savely removing, it would automatically close or release such handles to let the drive be safely removed.
    Are you saying that is the way it now works?
    Yes, if I remember it right (and have understood it right) it is what the developer says.


    Are you saying that is the way it now works? ...or that settings can now be tweaked to do that? I haven't checked the site for a while, and I'm running v1.4.0.713b, which I know is not the most recent version. I'll try to check for updates as soon as I get a chance.
    Yes, the new version is super. Keeps offline indexes automatically, indexes offline drives automatically when plugged in.

    If that is easy to replicate, or if you run into a similar situation, it would be nice to know for sure, as it might imply Recycle Bins are keeping the ...$Extend\$RmMetadata\$Txf handles open. Apparently it is established that a corrupted Recycle Bin is one cause for the handles not closing, I would then want to get some idea of how often that is the cause. What if it is the main cause, or even the only cause? I don't think that is the case as I don't think I'm likely to have corrupted Recycle Bins because I nearly always delete files completely (Shift+Delete) rather than sending them to the Recycle Bin
    I am pretty sure the handle / process were / was released after that bin error was removed. I would not know how to reproduce such behavior, if such situation, respectively when it occurs again I will let you know what happens.

    That one I use, too. But it is just flushing the cache, it does not effect / stop the journaling, NTFS actions, I would say.
    May be flushing sometimes causes Win to stop journaling / NTFS actions. May be it depends on the state the NTFS / journal is.

    That one I use, too. But it is just flushing the cache, it does not effect / stop the journaling, NTFS actions, I would say.
    Agreed, both Sync & FFB just flush the data in the cache to the drive, but theoretically that should stop any NTFS journaling as well, as that logging is done almost instantly before data is written, updated almost instantly while data is being written, and finalized almost instantly when the data-writing is completed (see discussion above).
    Yes. But it occurred that after wating for about 15, 20 minutes or so suddenly a drive could be removed safely after several tries. I would think it depends on what / how much Win writes, indexes, (re)builds, does regarding the journaling, NTFS.

    This one can be used, too: FFB.zip
    Ahhh, I see you have also found USB expert, Uwe Sieber (although you probably use his German page). I don't think I've visited his site since I was researching the ...$Extend\$RmMetadata\$Txf handles a few years ago, and I don't think I grasped the issues well enough at the time to have realized FFB might be useful. (...Might have even been before FFB was first introduced in April 2013 but probably since then.) ...But I had been thinking while drafting each of my posts in this thread that I would go back and check his site for anything new on the issue as soon as I get a chance.
    Yes, there are some programs connecting to safely removing / USB and such. It is worth a visit (in this context).

    According to Google, there is no mention of "$Extend\$RmMetadata\$Txf" (or "$Extend" or "$RmMetadata" or "$Txf") anywhere on his site, but he's the only one I've run across on the internet who I'm guessing could probably authoritatively clarify the ...$Extend\$RmMetadata\$Txf handles issue. I remember thinking about trying to email him about it a few years ago, but never got around to it.
    Yes, indeed, a good idea to ask him.

    I notice he mentions a couple of limitations of Sysinternals Sync that FFB overcomes:
    "The Sysinternals sync tool can flush volumes only which have a drive letter. And it forgets drive Z: when called without parameters"
    Yes, I remember to have read that as well. And FFB shows "OK" for a successfull execution, I am not sure anymore, but there shall be circumstances which let that sync tool fail flushing external / USB drives (without displaying it).

    It would be interesting to know if the FFB -f switch simply does the same thing as closing handles in Process Explorer:
    -f force dismount (open handles become invalid)
    Yes, would be difficult to test, I assume.

    If so, the switch would presumably force close all open handles on the drive. That, along with detection of what is preventing 'Safely Remove Hardware", would be the kind of function I think Microsoft should build in to automatically use (or manually selectable by user), if Windows detects that only the four ...$Extend\$RmMetadata\$Txf handles are preventing "Safely Remove Hardware".
    Yes, sounds like a good approach.

    Thank you very much for the explanations sounding very plausible.
      My ComputerSystem Spec

 
Page 3 of 3 FirstFirst 123


Similar Threads
Thread Forum
Cannot Safely Remove External Drives
Hello all, In recent weeks, it has become impossible for me to safely remove my external drives. (Mostly, I use Western Digital Passports, Passport Ultras, etc). I am using Windows 10. I close all of my programs and folders. I get the...
Drivers and Hardware
Cannot eject safely usb external drive
Hello, The problem of beeing unable to eject the usb drive safely still exists, the only way I found (temporarily) is to disable it in the device manager, but then the change asks to restart the computer, if I do not restart, the drive is unabled....
Drivers and Hardware
Cannot eject safely an external USB drive
Hello, Recently I bought one "WD Elements" 1TB external drive for data storage. When attachet it to the pc, it was found by the file explorer, it is already NTFS formated, one whole partition. On it found "WD Smartware pro trial" installer and an...
Drivers and Hardware
Solved How can I remove a Seagate External HDD safely on Windows 10? Solved
Please answer! I just need to be able to remove it so I can use it like a backup. :drool: --EDITED AFTER SOLVED-- Figured it out! Go to Control Panel/Hardware and Sound then right click on the external, and press Remove Device. Then unplug...
Drivers and Hardware
Not able to safely remove external harrdrive through notification icon
I'm using a Lenovo yoga 3 pro for the first time, with Windows 10 pre-installed, so I haven't made any changes or updates. The icon in the notification tray for Safely Remove Hardware is there, but when I click (left and right) on it, the external...
Drivers and Hardware
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd
All times are GMT -5. The time now is 04:44.
Find Us
Twitter Facebook Google+ Ten Forums iOS App Ten Forums Android App



Windows 10 Forums