Win 10 INTERNAL_POWER_ERROR crashes, involve amdxata.sys Stor Filter

bilateral · 07 Sep 2017

I have added the two files the log collector program gave me, though I'm not sure if these are what is wanted--I thought the reports would be more extensive. I will try to explain the problem briefly with the details that seem important and unique.

Frequent crashes in the last two weeks, sometimes with Blue Screen, sometimes only a black screen. When Win 10 displays Blue Screen, Internal_Power_Error may be shown. Who Crashed reports also amdxata.sys, the Stor Filter Driver as being involved, and usually ntoskrnl.exe.

Interesting is that few or no crashes have occurred during heavy usage--or any usage at all. They have been after user walked away and out of the room. But one at least before any power features like blanking the screen or requesting sleep came into play. No gaming is involved. But many Chrome user Tabs are generally open. Once or twice a blue screen has appeared, in one case on only one (the main) of two active monitors--other monitor retained image of the desktop, more often it is just two black screens, with the computer still running, never has a crash restarted the machine. In each case, the fans start to gear up after the crash, and the CPU gets very hot. Machine needs to be killed or reset. After restart, CPU cools down and fans slow down.

I am hoping for guidance on what to look for and test for. Who Crashed reports:

Windows version: Windows 10 , 10.0, build: 15063Windows dir: C:\WINDOWSHardware: GA-880GM-UD2H, Gigabyte Technology Co., Ltd.CPU: AuthenticAMD AMD Phenom(tm) II X6 1055T Processor AMD586, level: 166 logical processors, active mask: 63RAM: 12881264640 bytes total (That's 12 GB of DDR3, and a 6 core AMD processor.)

I don't know if this is likely to be a hardware problem. Should I test the memory overnight?
There is quite a bit of USB hardware, and also two monitors on the video card. Do you need a report of devices?

Or is it software? Chrome, many tabs and lots of complex web sites, often with video conferencing, is involved, but that was the case before recent crashes. Most crashes identifed the Stor Filter Driver in Who Crashed, but I don't think there are any updates for that. These are also since the last major update of Win 10, though there may have been a few rare crashes before that. Crashes have been about once or twice per day, sometimes a couple more.

Let me know what to do next, to try to narrow the problem.

Thanks,
Michael

Spectrum · 07 Sep 2017

There should be a .zip file that contains the logs, if you don't see one please run the log collector again.

bilateral · 08 Sep 2017

Okay, finally found it. Sorry, there seems to be two each of personal folders on this machine.

Spectrum · 08 Sep 2017

The problem reported in the 3 crash dumps, is that your AMD raid controller is trying to process a non-existent request. This is likely a driver issue, although it could mean the controller itself is failing.

Update to the latest chipset drivers from Gigabyte. If you have already tried that, see if you can update the driver from AMD's site.

After the driver has been updated, check your disks for errors using HDTune.

Then run chkdsk on all internal drives to ensure your filesystem is healthy. Open an admin cmd/powershell prompt and run "chkdsk /R /X <drive letter>" e.g. "chkdsk /R /X C:". Press Y if prompted to schedule a check on the next reboot, then reboot.

After chkdsk has completed post the resulting event log entries. Here is a guide on doing that: Read Chkdsk Log in Event Viewer in Windows 10 Windows 10 Performance Maintenance Tutorials

Below is the stack of the thread that called the BugCheck (Blue Screen). It's read bottom to top, as you can see amdxata is the driver sending a power IRP (I/O request packet) up to the Power Manager in the kernel.

We know this IRP is spurious due to the bugcheck code 0xA0 (Internal Power Error), and the first parameter, 613 (A driver has attempted to complete a request when no such outstanding request is pending).

Code:

00 ffffcb00`5c9262f8 fffff802`8ca33220 nt!KeBugCheckEx
01 ffffcb00`5c926300 fffff802`8c9c20b2 nt!PopFxBugCheck+0x1c
02 ffffcb00`5c926340 fffff802`8c93fe1f nt!PopFxCompleteDevicePowerRequired+0xe20f2
03 ffffcb00`5c926370 fffff80a`79d9d4dc nt!PoFxReportDevicePoweredOn+0x9f
04 ffffcb00`5c9263f0 fffff80a`79d9d3a7 storport!StorPortUnitPoFxD0Completion+0x64
05 ffffcb00`5c926470 fffff80a`79d9d271 storport!RaidUnitProcessSetDevicePowerIrpComplete+0x107
06 ffffcb00`5c926500 fffff80a`79d9d1b0 storport!RaidUnitProcessSetDevicePowerIrp+0xb9
07 ffffcb00`5c9265c0 fffff80a`79d9cf20 storport!RaidUnitSetDevicePowerIrp+0xec
08 ffffcb00`5c926620 fffff80a`79d9ce46 storport!RaidUnitSetPowerIrp+0x68
09 ffffcb00`5c926670 fffff80a`79da0557 storport!RaUnitPowerIrp+0x6a
0a ffffcb00`5c9266c0 fffff80a`79e216be storport!RaDriverPowerIrp+0x97
0b ffffcb00`5c926700 00000000`00000001 amdxata+0x16be
0c ffffcb00`5c926708 ffffa10b`7a768190 0x1
0d ffffcb00`5c926710 ffffa10b`7a88db40 0xffffa10b`7a768190
0e ffffcb00`5c926718 ffffa10b`7a88db40 0xffffa10b`7a88db40
0f ffffcb00`5c926720 00000000`00000020 0xffffa10b`7a88db40
10 ffffcb00`5c926728 fffff802`8c94094b 0x20
11 ffffcb00`5c926730 fffff80a`7a5572ca nt!IopPoHandleIrp+0x13
12 ffffcb00`5c926760 fffff802`8c878107 CLASSPNP!ClasspPowerUpCompletion+0x2aa
13 ffffcb00`5c926820 fffff80a`79d99728 nt!IopfCompleteRequest+0x117
14 ffffcb00`5c926950 fffff80a`79d9720d storport!RaidCompleteRequestEx+0x88
15 ffffcb00`5c926a20 fffff80a`79d96e7a storport!RaUnitScsiIrp+0x34d
16 ffffcb00`5c926ac0 fffff80a`79e218bf storport!RaDriverScsiIrp+0x5a
17 ffffcb00`5c926b00 00000000`00000001 amdxata+0x18bf

Furthermore there are a number event log entries that show the raid controller being reset or that IRPs sent to the disks timed out. A reset is sent to a device generally when it hasn't responded or is otherwise not behaving normally.

Here are two examples:

Code:

Event[559]:
  Log Name: System
  Source: storahci
  Date: 2017-06-21T19:01:33.378
  Event ID: 129
  Task: N/A
  Level: Warning
  Opcode: N/A
  Keyword: Classic
  User: N/A
  User Name: N/A
  Computer: Wendy
  Description: 
Reset to device, \Device\RaidPort0, was issued.

Code:

Event[249]:
  Log Name: System
  Source: Disk
  Date: 2017-06-21T17:01:36.535
  Event ID: 153
  Task: N/A
  Level: Warning
  Opcode: N/A
  Keyword: Classic
  User: N/A
  User Name: N/A
  Computer: Wendy
  Description: 
The IO operation at logical block address 0x2681230 for Disk 1 (PDO name: \Device\00000027) was retried.

bilateral · 08 Sep 2017

Thanks very much for looking at this and analyzing the results. The machine is in use right now, so I am trying to find a time to do what you request, and I'll get back to you as soon as I can.

I'll mention that the machine is not using RAID, though there is of course a RAID controller on the SATA. There are three attached drives. An SSD is the OS drive C: and a 1 TB HDD is the primary data drive. But there is also a 3rd drive hooked externally with a separate power supply. Normally it is always available, but if there is an AC line power failure (we get them occasionally and there has been one in the last 10 days) the power does not return until that drive is restarted. There is an auto backup running, but if the power is off on that drive, it will fail. Also, when the drive gets full, which it does, we have to manually delete old backups to make more room on the drive (I'm in process of finding another BU program)--so the backup could fail for either of those reasons, but only if the drive is off will the controller not find the drive. Still, the time of the BSODs does not seem related to the auto backup attempts, which are at the same time every day.

bilateral · 12 Sep 2017

@Spectrum--Reporting back on following your recommendations:

It was quite difficult to find the right Win 10 drivers for the RAID/SATA controller. (I was cautious about installing a complete chipset package, as other functions seem to be working, so I focused on controller drivers. --I did not want one that tries to install Catalyst video drivers.) Gigabyte does not list a Win 10 driver--I tried drivers for 7 and 8.1. Win 10 kept rejecting my attempts, even when I found supposed Win 10 drivers via AMD. 10 would keep telling me that the "best driver is already installed."

Finally I was able to use the AMD install package:amd-catalyst-15.7.1-sb-sata-ahci-win10-win8.1-win7.exe

After restarting, however, I found I had the same AMD SATA Controller driver as before--the one Win 10 keeps identifying as the correct driver, even though the package was listed as August, 2017: This is AMD 7/23/2012 Driver Version 1.3.1.43

I suppose it's possible that the AMD software did install a different driver, but upon restart Windows 10 insisted again on installing the driver it wanted. Nevertheless, even if that's what happened, a new install of the same driver might improve things.

On the other hand, the AMD install software did successfuly install a new driver that had not been there before at all--AMD PCI IDE Controller 6/28/2013, Driver Version 5.2.2.179

Also, during the install process, the software notified me that it was going to install an updated USB Stor Filter Driver. I had received an indication from Who Crashed that the Stor Filter Driver was involved in one of the crashes; however I don't recall this driver as being described as for USB. Your analysis of the dump files did not mention a Stor Filter driver, only the RAID Controller (presumably also the SATA Controller), but I'm pretty sure this driver was mentioned by Who Crashed.

Drive Error Checking:
I first ran Windows Error checking under Properties/Tools, to see what it would say. It told me that all three drives are okay and did not need to be checked. I then ran the error check anyway in Powershell (Admin), as you directed.

Drive C: (OS, SSD) and D: (Data Drive, HDD) checked out normally, with only minor file corrections. No bad sectors were detected. Those reports are below.

On the external drive G: it found no bad file records, but did find one unindexed file in Stage 2 and recovered it to original directory. After that, the surface scan on G: ran into a problem, which might be related to the fact that the drive is mounted externally with separate power supply. It is a 2 TB HDD, and it simply stopped at about 10% of the scan. When I looked, the drive was still powered, but a red indicator light showed it was no longer spinning. I may have to mount this drive into a PC with internal SATA cable and PS and try again. There was no report of a problem in PowerShell, which simply froze. And there is no report on this chkdsk in the Event Viewer either.

I'm not sure, given that result, whether it is necessary to try again to run that surface scan on G: which takes a very long time. The fact is, there have been no further crashes since I posted here--even before I installed the new driver.

Here are the reports on the two disk checks that succeeded fully (I don't understand why the text pasted from notepad loses its carriage returns??:

Code:

Checking file system on C:The type of the file system is NTFS.Volume label is Wendy-C-OS.A disk check has been scheduled.Windows will now check the disk.                         Stage 1: Examining basic file system structure ...Cleaning up instance tags for file 0x6c0b.  525568 file records processed.                                                         File verification completed.  12796 large file records processed.                                      0 bad file records processed.                                      Stage 2: Examining file name linkage ...  624634 index entries processed.                                                        Index verification completed.  0 unindexed files scanned.                                           0 unindexed files recovered to lost and found.                     Stage 3: Examining security descriptors ...Cleaning up 1949 unused index entries from index $SII of file 0x9.Cleaning up 1949 unused index entries from index $SDH of file 0x9.Cleaning up 1949 unused security descriptors.Security descriptor verification completed.  49534 data files processed.                                            CHKDSK is verifying Usn Journal...Usn Journal verification completed.Stage 4: Looking for bad clusters in user file data ...  525552 files processed.                                                                File data verification completed.Stage 5: Looking for bad, free clusters ...  11818082 free clusters processed.                                                        Free space verification is complete.Windows has made corrections to the file system.No further action is required. 116296703 KB total disk space.  68236544 KB in 272641 files.    181068 KB in 49535 indexes.         0 KB in bad sectors.    606763 KB in use by the system.     65536 KB occupied by the log file.  47272328 KB available on disk.      4096 bytes in each allocation unit.  29074175 total allocation units on disk.  11818082 allocation units available on disk.Internal Info:00 05 08 00 f9 e9 04 00 d4 1c 09 00 00 00 00 00  ................5c 02 00 00 df 79 00 00 00 00 00 00 00 00 00 00  \....y..........Windows has finished checking your disk.Please wait while your computer restarts.

Code:

Checking file system on D:The type of the file system is NTFS.Volume label is Wendy-D-FILES.A disk check has been scheduled.Windows will now check the disk.                         Stage 1: Examining basic file system structure ...  198400 file records processed.                                                         File verification completed.  3204 large file records processed.                                      0 bad file records processed.                                      Stage 2: Examining file name linkage ...  225230 index entries processed.                                                        Index verification completed.  0 unindexed files scanned.                                           0 unindexed files recovered to lost and found.                     Stage 3: Examining security descriptors ...Cleaning up 82 unused index entries from index $SII of file 0x9.Cleaning up 82 unused index entries from index $SDH of file 0x9.Cleaning up 82 unused security descriptors.Security descriptor verification completed.  13416 data files processed.                                            CHKDSK is verifying Usn Journal...  547863752 USN bytes processed.                                                            Usn Journal verification completed.Stage 4: Looking for bad clusters in user file data ...  198384 files processed.                                                                File data verification completed.Stage 5: Looking for bad, free clusters ...  75511167 free clusters processed.                                                        Free space verification is complete.Windows has scanned the file system and found no problems.No further action is required. 976755711 KB total disk space. 673818004 KB in 141584 files.     63248 KB in 13417 indexes.         0 KB in bad sectors.    829791 KB in use by the system.     65536 KB occupied by the log file. 302044668 KB available on disk.      4096 bytes in each allocation unit. 244188927 total allocation units on disk.  75511167 allocation units available on disk.Internal Info:00 07 03 00 ea 5c 02 00 c3 ac 03 00 00 00 00 00  .....\..........96 aa 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

zbook · 13 Sep 2017

1) Download and install:
Speccy - System Information - Free Download
2) Update the "My Computer" specifications:

How To Fill Out Your System Specs
How To Fill Out Your System Specs
System Specs - Fill in at Ten Forums:
System Specs - Fill in at Ten Forums Windows 10 General Tips Tutorials
In the left corner below in your post you find 'My System Specs'.
After clicking it you can find a link a little below that says 'Update your System Spec', click on this link to get to the page where you can fill in your system specs.
System Info - See Your System Specs - Windows 7 Help Forums

bilateral · 14 Sep 2017

@zbook:
You didn't specify why you want me to send this spec information, subsequent to what Spectrum requested--so I am not sure what you are looking for at this point.

I should add that at the moment there is little point in my filling in my system specs on my machine, because that is not the machine that was having the crash problem and which I requested input/assistance on. The problem machine is my wife's desktop computer. So I am attaching the Speccy text file for her computer. Is there anything further that stands out for you in her Speccy report?

Thanks,
Michael

zbook · 14 Sep 2017

The "My Computer" section allows other viewers to view a thread in post 5 or post 50 and not have to view all of the posts to find needed information about the underlying computer. There is an option to add more than 1 computer. If you can update this section you will make it easier for others to view and comment in the the thread. In addition having the component information available makes it easier to check component specifications for stress testing. Please indicate whether it is computer 1 or computer 2 so the specs can be matched to the thread.

Spectrum · 14 Sep 2017

bilateral said:

@Spectrum--Reporting back on following your recommendations:

It was quite difficult to find the right Win 10 drivers for the RAID/SATA controller. (I was cautious about installing a complete chipset package, as other functions seem to be working, so I focused on controller drivers. --I did not want one that tries to install Catalyst video drivers.) Gigabyte does not list a Win 10 driver--I tried drivers for 7 and 8.1. Win 10 kept rejecting my attempts, even when I found supposed Win 10 drivers via AMD. 10 would keep telling me that the "best driver is already installed."

Finally I was able to use the AMD install package:amd-catalyst-15.7.1-sb-sata-ahci-win10-win8.1-win7.exe

After restarting, however, I found I had the same AMD SATA Controller driver as before--the one Win 10 keeps identifying as the correct driver, even though the package was listed as August, 2017: This is AMD 7/23/2012 Driver Version 1.3.1.43

I suppose it's possible that the AMD software did install a different driver, but upon restart Windows 10 insisted again on installing the driver it wanted. Nevertheless, even if that's what happened, a new install of the same driver might improve things.

I'm glad to hear there haven't been anymore crashes, but if you do want to update the driver, you can likely manually select the .inf file from the extracted files the AMD driver installer throws on your C: drive via the device manager (or pnputil.exe).

bilateral said:

Here are the reports on the two disk checks that succeeded fully (I don't understand why the text pasted from notepad loses its carriage returns??:

Click the top left button (Switch WYSIWYG to Source mode), the default edit mode ruins a lot of formatting.