As a fellow SWE who's been in the RPA space for a few years now, I'll offer an alternative perspective.
Traditional RPA is here to stay, and only getting bigger. By "traditional" I mean screen-scraping and click bots. It's not only for legacy apps. It also addresses two development pain points that will never go away: (1) complexity and (2) missing features.
On (1), I've worked with companies whose APIs are so convoluted and poorly supported, and distributing your client in their ecosystem is so complex, that I've thrown up my hands and gone, "Forget it, I'll implement this with a service account." An RPA process logs into the front end, clicks around, scrapes some data, outputs it to the next process in my pipeline, and it's done. I've written processes like this that have been running for years with basically no maintenance due to stable-enough UIs. UI changes are still a risk, but if you have a mature UI, it's a great, simple alternative to a more complex process.
Regarding (2), APIs don't always expose all the features of the UI, and sometimes vendors won't, or can't, add them in a given budget or time frame. I worked with a partner whose API had essentially one read-only endpoint. Their product was fantastic and they had other integration methods; they just hadn't prioritized API development, which they could afford to do because they delivered so well in their niche otherwise. We had to get creative in how we would pull the data.
I know where you're coming from, but if the ROI is there, it doesn't matter if the first iteration is fragile.
Just make sure your monitoring and support are rock solid. If the RPA stumbles, someone needs to know right away, and they need to be there to catch it. And it should never be designed in a way that it could cause critical issues in the meantime.
I would add, RPA isn't as fragile as its reputation sometimes suggests anyway. Sure, using it on a top-500 site is going to be a problem because of frequent UI changes. But we've successfully used it on systems in less "move fast and break things" industries that have been there twenty years, and they're going strong.
Using it to monkey patch "features" onto external vendors is huge.
Sometimes, it's because the feature cuts against the vendor's business model (e.g. Verint's terrible CCTV server appliances / software client not allowing bulk export).
More often, it's because it's not a priority (e.g. mass-updating on a system that only allows a single update at a time).
Think a dedicated account in an old 3-tier architecture meant for a bot to login rather than a human. You're not making API calls though (at least not as we typically describe API calls), you're making HTTP requests to pull down a web page and then screen-scraping within the resulting HTML/JS.
Without a dedicated account your RPA bot needs to essentially impersonate a human or let the human login first and then drive the dashboard until the bot completes and the human can resume using their desktop seat. Very inconvenient for the human, and problematic for logging of who's really doing what.
Were it an internally owned application that's a legitimate course, but more often than not it's to grab data from third-party provider applications where DB is not an option, i.e. banking sites
ok, but normally banking has established protocols? If this is the main use-case of RPA-startups, Microsoft does not compete in any way. I don't see a lot of reasons (name one for me) for a legitimate business to pay an RPA-startup over the bank for automated data acquisition.
Another use-case could be the cloud probably – like scraping your salesforce apps. And this is just .... sad?
Traditional RPA is here to stay, and only getting bigger. By "traditional" I mean screen-scraping and click bots. It's not only for legacy apps. It also addresses two development pain points that will never go away: (1) complexity and (2) missing features.
On (1), I've worked with companies whose APIs are so convoluted and poorly supported, and distributing your client in their ecosystem is so complex, that I've thrown up my hands and gone, "Forget it, I'll implement this with a service account." An RPA process logs into the front end, clicks around, scrapes some data, outputs it to the next process in my pipeline, and it's done. I've written processes like this that have been running for years with basically no maintenance due to stable-enough UIs. UI changes are still a risk, but if you have a mature UI, it's a great, simple alternative to a more complex process.
Regarding (2), APIs don't always expose all the features of the UI, and sometimes vendors won't, or can't, add them in a given budget or time frame. I worked with a partner whose API had essentially one read-only endpoint. Their product was fantastic and they had other integration methods; they just hadn't prioritized API development, which they could afford to do because they delivered so well in their niche otherwise. We had to get creative in how we would pull the data.