How can I see if my internet drops out?

Hi, it seems as though the system tray wireless icon is more an indicator of there being a Wifi signal to a router, and not an indicator of internet access. Thus if I disconnect my ADSL connection, there is no change in the icon, whereas if I switch off the router, a small * appears next to the icon. I did briefly see a yellow triangle next to it.

Not an answer, just an observation for now- is that what you see?

On mine I get the yellow triangle if no internet access and router is on, if the router is off I get the * and no wifi signal

not sure if there is a setting for this, but will check my machine

Well, eventually with ADSL disconnected from my router, I got the yellow triangle. Say up to 2 minutes. So short disconnections wouldn't be indicated on that basis, even though there clearly was none.

Is that primarily Windows, or the router, or their interaction?

With ADSL disconnected, no internet connection at all, ping still works.. - yellow triangle on Wifi icon.
There is an open BT Wifi connection (odd!) in the area, but I'm not connected to it - apparently.

Ok so that was with one connection monitor program I've promptly binned.

Here's one that works.
Try Emco Ping Monitor (free). It's an enormous download for what it does (about 59Mb), but of 4 programs I tried, it's the only one that works. (I.e. tells me when I'm disconnected - ADSL - on the first ping). (Emco's a good company- see their 'Move on boot - also delete on boot etc for example).

Setup requires an email address to which notification would be sent (which is a bit weird, if the connection is down.. maybe it reports later?)

But you get voice, sound and a fly-in from the right notifying you of the connection failure.
Oh, and the tray icon changes colour.

Thinking now about addressing your basic problem of your Wifi dropping out, please try this simple fix:
Solved WiFi Internet Keeps Disconnecting - Windows 10 Forums
to disable a Win 10 feature.