If you have the headset plugged in and mute it, it won't work after you unmute regardless if you unplugged and re-plug in the headset.
No issue with the speaker if you unplugged the headset.
You may have to adjust the volume to zero and up the volume again for it to work again.