Can't access router.

Ok - here is how you configure two routers as you want.

First router downstairs is setup as per the ISP default. You need to know its IP address e.g. 192.168.1.1. No configuration of that router is needed.

Then the upstairs router needs to have an IP address in the same subnet i.e. the third number needs to be the same with fourth number different e.g. 192.168.1.2 (poster earlier said 192.168.2.1 which is wrong way round and would not work if just connecting lan ports).

The fourth number has to be outside the dhcp range so IP is static.

Finally, you need to turn off DHCP in upstairs router.

Your non-Comcast router needs to be 192.168.2.1 not 192.168.1.2. Otherwise it thinks that the Comcast Gateway is handing out a Static IP to it as a controlled device.