I have installed bunch of T620, they’re great. There’s also 4c version with AMD GX-415GA for ~60USD. If you need PCIe check out Fujtsu Futro s920[1]. You can fit a low profile network card with a 5USD riser. About <100 USD where I buy them.
GPU is more powerful, CPU is comparable. Similar power usage. OTOH RP4 doesn’t have builtin SSD, you need additional case, additional heat sink if you plan to actually use it for CPU, and it’s not x86. Some also take PCIe cards without extra boards/hacks. If you don’t need GPIO ports and just want a low power server/appliance then thin clients are still much better option. I use them for VoIP, HomeAssistant etc.
Well, it's a Pentium 4 with 2GB DDR2 RAM... You may wish to search around a bit.
https://www.ebay.com/itm/Dell-Optiplex-745-PC-SFF-Desktop-In...
HP T620 workstation ~$45 plus shipping?
https://www.ebay.com/itm/HP-T620-Dual-Core-AMD-Gx217Ga-1-65G...