Fixed it by going to regedit and setting DisablePreviewWindow to 0 under Advanced and setting NumThumbnails under Taskband to higher value, both of the folders were in...

I think this happened after I've set visual effects to "adjust for best performance" and set it back to custom and enabled all settings later on. Here are screenshots of settings i currently have...