You could also create a desktop shortcut with explorer.exe prefix and a space, change the icon then drag that to the taskbar.

Example:
explorer.exe %windir%\system32\mmsys.cpl