I worked for a very early billpay company where you could pay your bills online to vendors, even if the vendor didn't support it. We used API's where we could, but where we couldn't...
We had a whole team dedicated to keeping up the changes vendors would make to their websites that we scraped for info. The team was called, of course, "Scrape and P(r)ay".
If you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page.
So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
Depends on your development and per-action cost. And on the possible latency. It also changes your whole stack from "send a request" to "emulate each step in a browser while taking screenshots at (hopefully) the right event/delay" - that's a huge difference.
That's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine.
> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties.
Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
My wife hates it when 1099 her for services rendered, especially since she refuses to bill against the PO I set up for her. The kids at least accepted NET15 payment terms. Although the oldest said if I short pay him for lawn care again, he’s going to take away my early pay discount.
How does that work, though? Setting up a company has an initial cost and then recurrent costs (accountant, etc). Are the benefits that high for the average Joe?
It's hard to answer without specifics, even if you're not doing anything neferious there are a lot of benefits to putting an entity between yourself and your customers. It depends on where you live and what your business is of course.
