So, I reverted to Driver version 347.88 and most of these issues got fixed.
The latest driver that works without issue for me is 350.12 for Windows 8.1.
Yes - Using windows 8.1 drivers on windows 10 is more stable than the official drivers for Win10. Go figure.
I tested newer versions of windows 8.1 drivers, but none of them work and the windows 10 drivers are just broken on Nvidia Optimus
For now, there is no benefit of having a directx 12 driver, so I'd recommend sticking with the aforementioned version in order to have a slick Windows 10 experience.