MicroSD Card keeps partially overwriting existing files despite chkdsk

HedgeToaster · 2 Weeks Ago

I have a 400 GB SanDisk Micro SD Card that I use with my Android phone.

Well, actually it's my second one, I have two now.

Each has lasted about 2 years before developing a strange defect, wherein————
1) new files transferred to the phone from my PC (usually via USB connection)
2) photos shot with the phone in "quick" succession (1 - 3 seconds apart) as well as recorded videos
...would sometimes have some of those files or photos/videos be ~~corrupted~~.* Not all of them, not all the time, but often enough.

*EDIT 06.05.24: What I meant to say was "jumbled up with garbage data", not literally "corrupted". I.e., in case of image files, I often end up with stuff like this:

MicroSD Card keeps partially overwriting existing files despite chkdsk-img_20230411_034638a.jpg

...Although actual corruption does apparently occur also, it just isn't as noticeable. °shrug°

This was annoying, but manageable — just double-check new photos/videos and newly transferred files to see if any of them are broken and replace them.

When this happened with my first 400 GB SanDisk, I replaced it with a second 400 GB SanDisk.

That second card developed the same flaw after another 2 years — but now, after ~2.5 years, it's gotten worse.

Now, not only do new files sometimes get ~~corrupted~~damaged, but OLD files that had previously been functional get damaged when I transfer new files to the phone.

Basically, every time I copy a larger amount of new files to the card, I'm playing Russian roulette with the data that is already on it.

I've tried chkdsk.

This has yielded logs like this (heavily abbreviated, these logs can be 600+ lines long):

chkdsk T: /f
The type of the file system is exFAT.
Cannot lock current drive.

Chkdsk cannot run because the volume is in use by another
process. Chkdsk may run if this volume is dismounted first.
ALL OPENED HANDLES TO THIS VOLUME WOULD THEN BE INVALID.
Would you like to force a dismount on this volume? (Y/N) y
Volume dismounted. All opened handles to this volume are now invalid.
Volume Serial Number is [REDACTED]
Windows is verifying files and folders...

Cleaning up minor inconsistencies on the drive.
Corruption was found while examining files in directory \WhatsApp\Media\WhatsApp Video\ (98).
Cleaning up minor inconsistencies on the drive.
Corruption was found while examining files in directory \WhatsApp\Media\WhatsApp Audio\ (6).
Cleaning up minor inconsistencies on the drive.
Corruption was found while examining files in directory \Sounds\ (568).
Cleaning up minor inconsistencies on the drive.
Corruption was found while examining files in directory \Sounds\ (969).
Cleaning up minor inconsistencies on the drive.
Corruption was found while examining files in directory \Sounds\ (1758).

[...]

Corruption was found while examining files in directory \-DATA2\-TEXT\ (7777).
Corruption was found while examining files in directory \-DATA2\-TEXT\ (7781).
Corruption was found while examining files in directory \-DATA2\-TEXT\ (7783).
// There was, like, 100+ such messages in just that directory alone

[...]

Corruption was found while examining files in directory \-DATA2\-TEXT\ (8188).
Corruption was found while examining files in directory \-DATA2\-TEXT\ (8191).
Corruption was found while examining files and directories.
Corruption was found while examining the volume bitmap. // This seems relevant.
Windows is verifying file allocations...
\WhatsApp\Media\WhatsApp Documents\Sent\NEW TEXT-1.pdf is cross-linked on allocation unit 490386.
Cross link resolved by copying.
\Sounds\.vtdata\20211025_191634_text is cross-linked on allocation unit 1416990. // This too seems relevant.
Cross link resolved by copying.
\Sounds\.vtdata\20230227_162641_text is cross-linked on allocation unit 708206.
Cross link resolved by copying.
\Sounds\.vtdata\20230711_020018_text is cross-linked on allocation unit 559721.
Cross link resolved by copying.

[...]

\Sounds\.backup\[some long folder name] is cross-linked on allocation unit 754076.
Cross link resolved by copying.
\Pictures\Screenshots\Screenshot_20220723-142416.jpg is cross-linked on allocation unit 755958.
Insufficient disk space to copy cross-linked portion.
File being truncated. // That's a bit odd, there's at least 20+ GB left on the card, and the screenshot in question is 359 kb, it shouldn't trigger such a message.
\DCIM\Camera\IMG_20240306_155202.jpg is cross-linked on allocation unit 489902.
Cross link resolved by copying.
\DCIM\Camera\IMG_20240316_041925.jpg is cross-linked on allocation unit 488526.
Cross link resolved by copying.
\DCIM\Camera\IMG_20230718_043604.jpg is cross-linked on allocation unit 724605.
Cross link resolved by copying.
\DCIM\Camera\IMG_20230818_045200.jpg is cross-linked on allocation unit 724565.
Cross link resolved by copying.
\DCIM\Camera\VID_20240121_143541.mp4 is cross-linked on allocation unit 1409506.
Cross link resolved by copying.
\DCIM\Camera\IMG_20240121_143603.jpg is cross-linked on allocation unit 1409450.
Cross link resolved by copying.
\DCIM\Camera\VID_20240121_143646.mp4 is cross-linked on allocation unit 1410341.
Cross link resolved by copying.
\DCIM\Camera\IMG_20240219_012146.jpg is cross-linked on allocation unit 684717.
Cross link resolved by copying.
\DCIM\Camera\IMG_20240221_225151.jpg is cross-linked on allocation unit 559734.
Cross link resolved by copying.
\Android\data\com.google.android.apps.books\[some long folder name] is cross-linked on allocation unit 1064576.
Cross link resolved by copying.

[...] [...] [...]

\-DATA\-AUDIO\BOOKMARKS\bookmarks_28.06.23.mpb is cross-linked on allocation unit 752790.
Cross link resolved by copying.
File and folder verification is complete.

Windows has made corrections to the file system.
No further action is required.

384573440 KB total disk space.
363220736 KB in 91625 files.
460032 KB in 1777 indexes.
0 KB in bad sectors.
512 KB in use by the system.
20892160 KB available on disk.

262144 bytes in each allocation unit.
1502240 total allocation units on disk.
81610 allocation units available on disk.

From what I understand, the MicroSD card can't figure out that data is already present in that "unit" and it overwrites it to store new files, thus breaking the old files.

I've run chkdsk multiple times. Sometimes chkdsk will claim to have successfully repaired all errors, only to find more errors when I run it a second time immediately afterwards.

The problem persists.

Before I go out there and buy a new 400 GB Micro SD card, I'd like to ask the experts if this is somehow a "fixable" problem.

I know how hard drive defragmentation works; I know that a single large file isn't necessarily stored continuously in one location on the disk, but instead split into many pieces spread out across many sectors whereever there happens to be space available at the time.

I imagine something similar happens with files on a Micro SD card; and in the case of my particular Micro SD card(s) it can't seem to figure out which sectors are already in use and which are not, so it'll sometimes randomly overwrite data it shouldn't have.

What confuses me greatly is... well... how does that even happen???? Like, the "lookup table" of available sectors is broken or something? Except, right up to the point a file gets partially overwritten by new data, that file had remained perfectly readable. So... if the "lookup table" had been damaged, wouldn't that file have already been unreadable before?

And "intuitively", this seems like a problem that ought to be fixable somehow. Like, it's not like there are "physically damaged sectors" on the MicroSD card — just that the MicroSD card makes "strange decisions" about where to write new data.

Heck, I can even tell that the problem is "spreading outwards" ——— groups of pictures copied to the phone at the same time that initially only had 1 or 2 ~~corrupted~~ damaged images now have more

Also, since both SanDisk cards purchased 2 years apart had the same defect, I'm guessing SanDisk isn't... known for the quality of its products. Any suggestions on a cheap 400 GB Micro SD card from a better manufacturer?

cereberus · 2 Weeks Ago

SD cards are more prone to failure than normal drives. Worse many are fakes. Frankly I would not trust any high capacity SD card.

Buy an ssd and put it in a usb enclosure. It will be cheaper.

Then for phone, use a much smaller capacity card e.g. 64 GB, but offload data to usb drive (via a pc preferably so you can backup data).

For large volumes of data on a mobile phone, frankly better to use a cloud service.

HedgeToaster · 1 Week Ago

cereberus said:

Buy an ssd and put it in a usb enclosure. [...]
Then for phone, use a much smaller capacity card e.g. 64 GB [...]
[...] frankly better to use a cloud service.

Ooof. I could list several reasons why that really really is not an option for me, but yeah, sorry, an external storage device or a cloud storage won't work for me.

And going down to 64 GB is ... well, uhm. I'm already running low on my 400 GB card as it is. These aren't "some random files I carry around and look at once in a blue moon", I use these every day, and certain apps need to access them directly. Heck I could use an upgrade to a 512 GB if I end up buying a new one. So... sorry, but not an option for me. O_o

Not everyone uses their phones the same way, just as not everyone does with their PCs. Everybody has a different use case. Yours apparently differs from mine significantly. I hope I did not offend. ^_^;;;

Before I buy a new card, I want to first know if it is at all possible to fix my existing Micro SD card(s).

Say, try formatting the card and putting all the data back on it afterwards (which I've so far avoided), or try different chkdsk parameters than the one I used ( "chkdsk T: /f" ). Luckily, the cards are still pretty much fully accessible.

As for buying a new card ... well, I've looked into it, and apparently the new "A2" standard which is meant to be faster actually performs worse than A1 on older phones like mine.

Darn, I was really eying up that 35€ Samsung 512 GB card, but no, that's an A2.

That somewhat limits my options. Any suggestions what brands of MicroSD card won't start damaging files after 2 years?

^{PS: Also check above, I've updated the original post a bit to clear up some possible confusion, I used the word "corruption" when I actually meant something else.}

Samuria · 1 Week Ago

What file system is on card if it's fat32 there is a fault called looping fat as it exceeds the limits

cereberus · 1 Week Ago

No offence taken.

In the end, you get what you pay for. Cheap and quality is pretty much exclusive.

HedgeToaster · 1 Week Ago

Samuria said:

What file system is on card if it's fat32 there is a fault called looping fat as it exceeds the limits

When I connect the 400 GB Micro SD card to my PC via USB adapter, it reads as exFAT.

Berton · 6 Days Ago

There's 3 choices of formatting in Windows, NTFS for most drives, exFAT is good if sharing with Mac OS and lastly there is FAT32. All of my USB Thumb drives up to 256GB have come factory-formatted as FAT32 but Windows can't do FAT32 on drives larger than 32GB. FAT32 also has a single-file size limit of 4GB but NTFS and exFAT don't. As for SDHC and microSD/XD cards, I've found it better if they are to be used in a camera to format them in the camera, sometimes a computer-formatted card doesn't work in a camera even though both will be FAT32, I have SanDisk 8GB cards in my cameras formatted as FAT32. Windows can read and write to larger FAT32 drives.

JLArranz · 6 Days Ago

I think both cards have developed a failure in the wear levelling mechanism. If so, both cards are broken, the same as if other inner part had broken instead.

Wear leveling - Wikipedia EEPROM and flash memory media have individually erasable segments, each of which can be put through a limited number of erase cycles before becoming unreliable. This is usually around 3,000/5,000 cycles[5][6] but many flash devices have one block with a specially extended life of 100,000+ cycles that can be used by the Flash memory controller to track wear and movement of data across segments. Lets suppose that when the first number is 4000 cycles and second one 100,000, both parts get worn out at a time (approximately). Now if the maker uses "80,000 cycles memory" instead, wear levelling will start to do wrong physical memory mappings to the LBA map when the user data cells are still in good shape. If "120,000 cycles memory" is used instead, nearly all units will develop user data corruption before. The same if "100,000 cycles memory" is used but the regular cells last more or less than 4000 cycles.

MisterEd · 6 Days Ago

First make a backup all data on the microSD card. Then download and run H2testw. H2testw will copy 1GB files to the card to its rated capacity and verify the copied data. The test will take a long time so be patient.

There are two possibilities if the card fails running H2testw. 1) the card has gone bad, and 2) the real size is less than the rated one. In either case the card is not to be trusted.

Download H2testw
H2TestW- Download

Here are the results of testing a 32GB Sandisk Ultra microSD card. It took about an hour to test it.

MicroSD Card keeps partially overwriting existing files despite chkdsk-h2testw.jpg

HedgeToaster · 5 Days Ago

JLArranz said:

I think both cards have developed a failure in the wear levelling mechanism. If so, both cards are broken, the same as if other inner part had broken instead.

Wear leveling - Wikipedia EEPROM and flash memory media have individually erasable segments, each of which can be put through a limited number of erase cycles before becoming unreliable. [...]

Ah, okay, that makes some sense, thanks for the explanation.

MisterEd said:

There are two possibilities if the card fails running H2testw. 1) the card has gone bad, and 2) the real size is less than the rated one. In either case the card is not to be trusted.

Thanks, I'll try that once I'm done with backup and backup verification.

In the meanwhile, I've been simultanously also looking into possible Micro SD cards to buy on Amazon in case it turns out the card(s) is/are unsalvagable.

Since I'm limited to A1 cards (A2 cards apparently actually perform worse than A1 on older phones like mine), and discounting anything without "a large number of reviews", the only A1 Micro SD cards in the 400 GB - 512 GB range I've found that seem reputable have been "kingston" and, well, San Disk again. Both currently offering 512GB cards for 36,99€. (165.000+ and 64.000+ reviews respectively, otherwise I'd assume foul play)

Looking through the kingston reviews filtered by "512 GB card" specifically seems to have a 90-10 positive/negative rating... except the negative 512GB card reviews are from people whose cards failed completely from one day to the next within 3 weeks to 6 months. (Oddly, mostly Steam Deck users instead of Smart Phone users though.) Of course there's positive reviews of people having years of positive experiences with kingston micro SD cards (not restricted to filtering by 512GB), too, which I found by searching for the keyword "years" and "Jahre" (German for "years"). Coin toss, as always.

Hmm. Better SanDisk, "the devil I know", which will probably last another 2 - 2.5 years before slowly developing the same fault, or try kingston? Choices, choices. :/