Thanks for the screenshot.

That's actually the Photos app.

Your title says:
Ridiculous problem viewing pics in MS File Explorer

- but actually from your post above, the rt arrow/Next belongs to the Photos app.

So when you say
File Explorer now goes through the all image files of the subfolder that a particular photo is in when I click on the arrow on the right side
what you're actually referring to is the behaviour of the Photos app.

I can only comment that if I rt click an image file in a folder, and open that with the Photos app (it's not my default viewer) the rt arrow behaves sensibly- 1 click = next image.