It certainly does delete them if you use PowerShell with administrator privileges. I've had issues with Hyper-V Manager myself, but didn't find out the reasons.

more discussion about checkpoints...