KERNEL_SECURITY_CHECK_FAILURE @ anniversary update, not before

We had a PC running GA-H55M-D2H mobo, 1x2GB DDR3 and Windows 7 32bit.
We upgraded (not fresh) to Windows 10 32bit.
We added 1x4GB DDR3. To this point, everything is fine, memory test perfect.
Now we need the Windows anniversary update to get our xbox controllers working, but every time we install the update, a restart is needed - this leads immediately to a BSOD claiming KERNEL_SECURITY_CHECK_FAILURE. The upgrade is reverted. I found no events or dumps when I checked.
We tried checking for viruses, sfc, chkdsk – no change.
We installed identical 2x4GB DDR3 – no change.
We added a discrete video card instead of the integrated – no change. (we purchased all hardware together)
A fresh install of current (downloaded a few days ago) Windows 10 64bit or 32bit, on a brand new clean unformatted drive:
If we run its memory test, then everything is perfect.
If we choose to install, it starts with filling a bar at the bottom twice, shows the Windows logo for 1 second, then BSOD again, before even asking where to install. Installing under safe mode or other options ended the same way.
I used minimal hardware with just a wireless adapter for keyboard & mouse.
Trying the same install with just the 1x2GB RAM - everything is fine, Windows installed, game & controllers working, no crushes.
BIOS is the latest, F4, and set to safe defaults.
4GB RAM: KVR16N11S8/4

Why won't the anniversary update run with the 4GB DDR3? - The older Windows builds are perfectly fine with them.
Is there any way to circumvent this? Disable the check?
We need more RAM to run games, and we need the controllers to work. I couldn't find any answer.
Unfortunately, I can't return the hardware, and I have neither other desktop DDR3 nor another desktop supporting it.

As the PC is away from me, I would only be able to check it on weekends. Please just tell me everything that I should do/check then.

Thanks in advance for any help or thoughts! :)

Thank you Boris. Sure, I will try all of that as soon as I am there. I didn't have that guide the last time.

Yes, I am aware that identical 2x4GB running a 64bit OS would be better, and that is what I was trying to do. I only checked 32bit again to verify that it is not a 64bit compatibility issue, or that some file is corrupt on that installation. So it's not that.

Dup

I don't have the files because I forgot copying them, but there were no dumps, and the tests were outside Windows anyway.
The other results:

Single 4GB – reboots a few times and stops.
Other combinations – boot nicely.

Memtest86+ 5.01 :
2GB – no problems
Combinations – test freezes during test #2 [Address test. Own address Parallel]. It reports no errors.
4+4 @ 47% Testing: 4096M-6144M 2048M of 8120M
4+2 @ 61% Testing: 4096M-6080M 1984M of 6072M
2+4 @ 64% Testing: 4096M-6080M 1984M of 5944M
The mix showed the same numbers as the 2GB – 666MHz BCLK:133 CAS 9-9-9-24, only 128-bit instead of 64-bit. Or whatever I manually set them to.

Windows 10 memory test – still finds no problems.

Discrete/integrated graphics makes no difference.
Installed drives make no difference.

Tried slower speeds and higher latencies, as far as I understand them – I only managed making things worse/same.

Any ideas?

I left the PC with the 2GB only, so it updated the old Windows with no problem.
Time passed and meanwhile I managed to replace the 2x4GB RAM with different modules of same specs, except low voltage - this.
Now the old 32 bit Windows boots fine with no errors, but the new 64 bit Windows still crashes before loading anything. I tried several times.
All other tests and configurations behave exactly the same as before, with the other RAM.
Both RAM models do work in a different PC.
I am not sure whether the different RAM made the difference or if it was some update to the 32 bit OS.
Do you have any thoughts?