Old Notepad Bluish colored edges

I guess it was discussed somewhere in the big Theme Thread.

It seems the Aero interface doesn't allow to change it.
That border is part of the buttons and buttons have a blueish highlight.
You can add a border over it, with a customized theme, but the blue one remains.

In High Contrast themes it is possible to change them all.

See here:
Solved Windows 10 Themes created by Ten Forums members - Page 128 - Windows 10 Forums

Yes it's normal.
On the other hand if you switch to a high contrast theme, you'll have control to all colors but will look less normal in some programs.