This sounds silly but I found a solution in another forum. Even if the "show all windows in task bar" is already checked uncheck it hit apply, then recheck that box.

Go into Excel Options >...