Extending LAN With third router/switch

Yes. (I understand that the downstairs access point is a router configured to act as an ethernet switch and wireless access point.)

In the LAN IP address range of the modem/router but outside the Dhcp server's address range and not the same as assigned to any other device on the LAN.

Connect a small ethernet switch (example) and you have the same functionality as with a router with built-in ethernet switch.