win 10 1903 "REFERENCE_BY_POINTER"

pstoric · 23 Nov 2021

I have a user that had a few BSOD’s happen yesterday but after remoting in and looking over the machine, I can seem to figure out what’s causing it.

It’s a Dell Inspiron running Win 10 1903 (10.0.18362). I tried to update it to 21H2 but it kept failing
SFC /scannow showed it fixed some errors but not all

DISM check came back clean
Drivers/BIOS were up to date
No third-party AV was in use

I ran windbg on the dmp file and it doesn’t seem to be a driver issue as far as I can tell. Any idea what can be causing it? All the BSOD's started on 11/22 it seems

************* Path validation summary **************Response Time (ms) Location

Deferred srv*

Symbol search path is: srv*

Executable search path is: 

Windows 10 Kernel Version 18362 MP (4 procs) Free x64

Product: WinNt, suite: TerminalServer SingleUserTS

Edition build lab: 18362.1.amd64fre.19h1_release.190318-1202

Machine Name:

Kernel base = 0xfffff806`49400000 PsLoadedModuleList = 0xfffff806`49843290

Debug session time: Mon Nov 22 19:46:29.725 2021 (UTC - 5:00)

System Uptime: 0 days 0:34:36.916

Loading Kernel Symbols

...............................................................

................................................................

..........................................................

Loading User Symbols

Loading unloaded module list

................

For analysis of this file, run !analyze -v

nt!KeBugCheckEx:

fffff806`495bc810 48894c2408 mov qword ptr [rsp+8],rcx ss:0018:ffff9203`799335f0=0000000000000018

3: kd> !analyze -v

ERROR: FindPlugIns 8007007b

*******************************************************************************

* *

* Bugcheck Analysis *

* *

*******************************************************************************



REFERENCE_BY_POINTER (18)

Arguments:

Arg1: 0000000000000000, Object type of the object whose reference count is being lowered

Arg2: ffffc286c2271050, Object whose reference count is being lowered

Arg3: 0000000000000010, Reserved

Arg4: 0000000000000001, Reserved

The reference count of an object is illegal for the current state of the object.

Each time a driver uses a pointer to an object the driver calls a kernel routine

to increment the reference count of the object. When the driver is done with the

pointer the driver calls another kernel routine to decrement the reference count.

Drivers must match calls to the increment and decrement routines. This BugCheck

can occur because an object's reference count goes to zero while there are still

open handles to the object, in which case the fourth parameter indicates the number

of opened handles. It may also occur when the object's reference count drops below zero

whether or not there are open handles to the object, and in that case the fourth parameter

contains the actual value of the pointer references count.



Debugging Details:

------------------





KEY_VALUES_STRING: 1



Key : Analysis.CPU.mSec

Value: 3093



Key : Analysis.DebugAnalysisManager

Value: Create



Key : Analysis.Elapsed.mSec

Value: 4449



Key : Analysis.Init.CPU.mSec

Value: 249



Key : Analysis.Init.Elapsed.mSec

Value: 2751



Key : Analysis.Memory.CommitPeak.Mb

Value: 72



Key : WER.OS.Branch

Value: 19h1_release



Key : WER.OS.Timestamp

Value: 2019-03-18T12:02:00Z



Key : WER.OS.Version

Value: 10.0.18362.1





FILE_IN_CAB: 112221-97531-01.dmp



BUGCHECK_CODE: 18



BUGCHECK_P1: 0



BUGCHECK_P2: ffffc286c2271050



BUGCHECK_P3: 10



BUGCHECK_P4: 1



BLACKBOXBSD: 1 (!blackboxbsd)





BLACKBOXNTFS: 1 (!blackboxntfs)





BLACKBOXPNP: 1 (!blackboxpnp)





BLACKBOXWINLOGON: 1



CUSTOMER_CRASH_COUNT: 1



PROCESS_NAME: System



LOCK_ADDRESS: fffff8064985d680 -- (!locks fffff8064985d680)



Resource @ nt!PiEngineLock (0xfffff8064985d680) Exclusively owned

Contention Count = 24

Threads: ffffc286c6dc9040-01<*> 

1 total locks



PNP_TRIAGE_DATA: 

Lock address : 0xfffff8064985d680

Thread Count : 1

Thread address: 0xffffc286c6dc9040

Thread wait : 0x2073a



STACK_TEXT: 

ffff9203`799335e8 fffff806`495d8af0 : 00000000`00000018 00000000`00000000 ffffc286`c2271050 00000000`00000010 : nt!KeBugCheckEx

ffff9203`799335f0 fffff806`49469876 : ffffc286`c2271050 00000000`69706e50 ffffae01`cb600000 ffff9203`000007cb : nt!ObfReferenceObjectWithTag+0x1445f0

ffff9203`79933630 fffff806`49a43a28 : 00000000`c00000bb ffff9203`79933710 ffff9c09`c038b580 00000000`00000000 : nt!IoGetAttachedDeviceReferenceWithTag+0x36

ffff9203`79933660 fffff806`49af9eae : 00000000`c00000bb ffff9203`799338d0 ffffc286`beedbd20 00000000`00000000 : nt!IopSynchronousCall+0x44

ffff9203`799336e0 fffff806`49af9d90 : ffffc286`bee8fb20 ffff9203`799338d0 ffff9203`799338b0 fffff806`49a8120a : nt!PnpIrpQueryID+0x56

ffff9203`79933770 fffff806`49af7615 : ffffc286`bee8fb20 ffff9203`799338d0 00000000`00000000 00000000`00000000 : nt!PnpQueryID+0x34

ffff9203`799337d0 fffff806`49af4f53 : ffffc286`bee8fb20 ffffc286`bee8fb20 00000000`00000001 00000000`00000000 : nt!PiProcessNewDeviceNode+0xed

ffff9203`799339a0 fffff806`49c72612 : ffffc286`bee8fb00 ffffc286`c9bf8801 ffff9203`79933ab0 fffff806`00000000 : nt!PipProcessDevNodeTree+0x3fb

ffff9203`79933a60 fffff806`4965c94c : 00000001`00000003 ffffc286`bee8fb20 ffffae01`00000000 ffffc286`c9bf88f0 : nt!PiRestartDevice+0xbe

ffff9203`79933ab0 fffff806`494ae835 : ffffc286`c6dc9040 ffffc286`bee7d260 fffff806`4985bf20 ffffc286`c6dfc370 : nt!PnpDeviceActionWorker+0x10e1ec

ffff9203`79933b70 fffff806`49530925 : ffffc286`c6dc9040 00000000`00000080 ffffc286`bee7f040 00002425`bd9bbfff : nt!ExpWorkerThread+0x105

ffff9203`79933c10 fffff806`495c3d5a : ffffae01`cb379180 ffffc286`c6dc9040 fffff806`495308d0 00000000`00000001 : nt!PspSystemThreadStartup+0x55

ffff9203`79933c60 00000000`00000000 : ffff9203`79934000 ffff9203`7992e000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x2a





SYMBOL_NAME: nt!ObfReferenceObjectWithTag+1445f0



MODULE_NAME: nt



IMAGE_NAME: ntkrnlmp.exe



IMAGE_VERSION: 10.0.18362.30



STACK_COMMAND: .cxr; .ecxr ; kb



BUCKET_ID_FUNC_OFFSET: 1445f0



FAILURE_BUCKET_ID: 0x18_nt!ObfReferenceObjectWithTag



OS_VERSION: 10.0.18362.1



BUILDLAB_STR: 19h1_release



OSPLATFORM_TYPE: x64



OSNAME: Windows 10



FAILURE_ID_HASH: {b6b47bfd-ed1b-fc56-3b01-ddc037f9c59f}



Followup: MachineOwner

---------

Win10Warlord · 24 Nov 2021

Please remove any Dell diagnostics software or Dell software in general which you may have installed. It appears to be causing problems.

pstoric · 24 Nov 2021

Win10Warlord said:

Please remove any Dell diagnostics software or Dell software in general which you may have installed. It appears to be causing problems.

Just curious, can you highlight what in the logs is pointing to any Dell software?

Win10Warlord · 24 Nov 2021

Code:

0: kd> !devobj ffff970ceb359050
Device object (ffff970ceb359050) is for:
 Cannot read info offset from nt!ObpInfoMaskToOffset
 \Driver\DDDriver DriverObject ffff970ce785bb50
Current Irp 00000000 RefCount 0 Type 00000022 Flags 00002040
SecurityDescriptor ffff860d2b2ef620 DevExt ffff970ceb3591a0 DevObjExt ffff970ceb361590 
ExtensionFlags (0x00000012)  DOE_DELETE_PENDING, DOE_START_PENDING
Characteristics (0x00000100)  FILE_DEVICE_SECURE_OPEN
Device queue is not busy.

DDDriver.sys is for Dell Diagnostics software. I've seen it cause similar problems in the past. Of course, you'll be able to obtain more information with a kernel memory dump (available at %systemroot%\MEMORY.DMP) but the issue does appear to be with Dell.

pstoric · 24 Nov 2021

Thanks! I'll get that removed later today. Also I posted on another forum and someone pointed out there may be an issue with the IME driver or firmware so I'm going to try to update or remove and reinstall that to see if it gets resolved. I'll post back my results.

The machine is in a remote location so I can only use my RMM tools, can't check it out in person.

Code:

Event[110]:
  Log Name: System
  Source: MEIx64
  Date: 2021-11-23T15:57:14.882
  Event ID: 3
  Task: N/A
  Level: Error
  Opcode: N/A
  Keyword: Classic
  User: N/A
  User Name: N/A
  Computer: DESKTOP-CM5H0FJ
  Description:
Intel® Management Engine Interface driver has failed to perform handshake with the Firmware (FWSTS0: 0x1E000042, FWSTS1: 0x66002106).

Win10Warlord · 24 Nov 2021

Okay, please let me know if you have any other issues.

pstoric · 24 Nov 2021

I just removed all Dell software and updated the IME driver remotely but after rebooting it seems like it crashed as it's been offline for 30 min so far. Not sure of they user is home but doesn't seem to have worked. Going to look for more crash dumps after they get it back online if possible

Win10Warlord · 24 Nov 2021

If it may worthwhile getting the kernel memory dump as well if possible, it will too large to upload as a post attachment, so you'll need to zip it and then upload it to a file sharing site such as Google Drive.

hsf · 21 Dec 2021

Win10Warlord said:

Code:

0: kd> !devobj ffff970ceb359050
Device object (ffff970ceb359050) is for:
 Cannot read info offset from nt!ObpInfoMaskToOffset
 \Driver\DDDriver DriverObject ffff970ce785bb50
Current Irp 00000000 RefCount 0 Type 00000022 Flags 00002040
SecurityDescriptor ffff860d2b2ef620 DevExt ffff970ceb3591a0 DevObjExt ffff970ceb361590 
ExtensionFlags (0x00000012)  DOE_DELETE_PENDING, DOE_START_PENDING
Characteristics (0x00000100)  FILE_DEVICE_SECURE_OPEN
Device queue is not busy.

DDDriver.sys is for Dell Diagnostics software. I've seen it cause similar problems in the past. Of course, you'll be able to obtain more information with a kernel memory dump (available at %systemroot%\MEMORY.DMP) but the issue does appear to be with Dell.

Hi there
Im having the same issue as well, its amazing you got the culprit however could you tell me how you got to "\Driver\DDDriver DriverObject ffff970ce785bb50" from the dump files cause Im trying to find that on mine and cant find it.
Thanks

pstoric · 21 Dec 2021

It worked for me