If this many are public right now, what does that say about the dark matter of private ones? What's the typical public-private rate for this sort of thing/can someone help me calibrate my base rate expectations?
Playwright, the end-to-end testing framework for the web, provides a strong incentive to give sites good a11y: Playwright tests are an absolute delight to read, write and maintain on properly accessible sites, when using the accessibility locators. Somewhat less so when using a soup of CSS selector and getByText()-style locators.
One thing I am curious about is a hybrid approach where LLMs work in conjunction with vision models (and probes which can query/manipulate the DOM) to generate Playwright code which wraps browser access to the site in a local, programmable API. Then you'd have agents use that API to access the site rather than going through the vision agents for everything.
I've mentioned several times and gotten snarky remarks about how rewriting your code so it fits in your head, and in the LLM's context helps the LLM code better, to which people complain about rewriting code just for an LLM, not realizing that the suggestion is to follow better coding principles to let the LLM code better, which has the net benefit of letting humans code better! Well looks like, if you support accessibility in your web apps correctly, Playwright MCP will work correctly for you.
There is also Testing Library, which I’ve mostly seen and used for unit tests (vitest) and component tests (Storybook), that practically forces you into setting things up in an accessible way. The methods for finding elements are along the lines of “find by ARIA role” or “get by label” - in fact, querying the DOM with selectors is afaik either not a part of the library or very difficult to do because their focus is ensuring your app is actually accessible as part of your testing strategy.
Was looking for this comment. I'd like to see this approach in the comparison...having the LLM build a playwright script and use it. I suspect it would beat time-to-market for the api, and be close-ish in elapsed time per transaction.
Harder to scale if it's doing a lot of them, I suppose.
I've found that by far the most useful websites as a programmer are also the ones most resistant to AI. This would be a huge loss for anyone vision impaired
What sorts of sites are you thinking of? To me, “most useful to a programmer” evokes docs and blogs and github issues and forum posts. I suppose some forums might be AI-resistant (login wall), but the others are trivially AI accessible.
That's less a value judgment, more a necessary evil due to the plethora of bad actors out there. I doubt it will get in the way of a local model used in a reasonable manner.
Most wikis you can mirror locally if you really need to hammer them.
And lets not forget that not all disabilities are chronic. Many disabilities are situational or temporary. AI is a great assist for a hangover day for example...
as someone who doesnt do web stuff, i found some humor in having no idea what "a11y" was, having to look it up, and finding out it is supposed to be "accessibility".
my quick accessibility tip: introduce what your acronyms, initialisms, and numeronyms stand for at least once.
a11y is pretty pervasive and well understood in the context around what is being discussed. I18n as well, you get to look that one up to because that makes you one of today's lucky 10000 https://xkcd.com/1053/
I mean…I guess. But this is ridiculous - how many layers does our technology need to bash through to update two records on remote systems? I get that value is being added at some point - but just charge some micropayment for transactions. This is just too much.
What's gonna really be funny is the first time a state legislates that an AV company has to keep a bug in their software to maintain a municipal income flow.
Twin Cities, 2010-2014: 95 pedestrians killed in 3,069 crashes. 28 drivers were charged and convicted of a crime, most often a misdemeanor ranging from speeding to careless driving. ~70% of pedestrian-killing drivers faced no criminal charge[0].
Bay Area, 2007-2011 (CIR investigation): sixty percent of drivers that were at fault, or suspected of being at fault, faced no criminal charges. Over 40 percent of drivers charged did not lose their driver's licenses, even temporarily[1].
Philadelphia, 2017–2018: just 16 percent of the drivers were charged with a felony in fatal crashes[2].
Los Angeles, 2010–2019: 2,109 people were killed in traffic collisions on L.A. streets... and nearly half were pedestrians. Booked on vehicular manslaughter: 158 people. The vast majority of drivers who kill someone with their car are not arrested[3].
I can literally do this all day. The original statement was correct, the case representative.
Now we’re talking. So much misinformation in this thread. There’s a reason that the saying, “if you want to kill someone, do it with a car” exists. Fortunately, it seems like judges are finally starting to wake up to the idea that it’s unreasonable for drivers to claim ignorance about the increased risks (and thus intent) of making poor/illegal decisions when being the wheel.
This thread talks about driverless cars; vehicular manslaughter requires negligence or intent, do you want to find narrowed statistics for driverless cars that are restricted to negligence or intent?
Criminality is basically just a checkbox for this stuff. Most of the time people wouldn't be going to jail for these sorts of crimes, it'd just be big fines and penalties. There's almost always administrative/civil infractions of the same or similar name that has the same or greater punishment but are far more efficient for the state to prosecute because the accused has fewer rights.
It makes for good appeal to emotion headlines to say these people aren't getting charged with crimes, but that's only half the story. They're likely lawyering up and pleading to a civil infraction that has approx the same penalties.
And this is true not just for this issue but for many subject areas of administrative law. Taxes, SEC, environmental, etc, etc, all operate mostly like this.
It's easy for a writer to pander to certain demographics and get people whipped into a frenzy by writing an easy article about prosecuting rates using public data. Actually contacting these agencies and figuring out what they actually did is hard and in the modern media economy doesn't offer much upside for the work.
Someone (i forget who) wrote that if someone invented a technology equally beneficial and equally harmful it wouldnt even be considered today but 100 years ago they wouldnt even question it. It was labor as usual.
Personally i would like to see a more granual permission to drive based on performance, need and demography.
Well, if the law treats them differently when it comes to punishment, then maybe it should treat them differently when it comes to being able to drive in the first place?
Yup. And we do have some degree of safeguards here-- admittedly, less in California than many other states. They are: physician required reporting of disqualifying conditions, ability for other people to report concerns about capability to drive, and the requirement to show up and undergo vision testing and not flag other concerns in the process.
There's a tradeoff between reducing the very low rate of unsafe driving by the elderly and the burden added to the very old. People over 65+ are still possibly safer, overall, than teenagers.
You noticed that too huh? It's weird ... It's not like they have to do this? They aren't forced to go full evil company mode by any extrinsic thing but even the way they frame it "welfare trap" trap? for whom?
Anthropic is actually trying to do some research into model welfare which I am personally very happy about. I absolutely do not understand people who dismiss it ... wouldn't you like to at least check? doesn't it at least make sense to do the experiments? ? Ask the questions so that we don't find out "oops, yeah we've been causing massive amounts of suffering" here in 10 years? Maybe makes sense to do a little upfront research? Which to be clear this paper is not.
Full disclosure: I didn't figure this out myself, I got it from Ms. Vale's review.
I agree that the term "welfare trap" is a loaded one. This looks to me to be a case of refusing to look through the telescope in case they might see something they do not want to.
Everybody's arguing about how silly this paper is (it is) and not grappling with the purpose of the paper. The purpose of the paper is what it does. This particular paper is perfectly-produced to show up when people type in AI consciousness fallacy to Google (try it!) it's something that anybody who has read a Freshman philosophy textbook will realize is silly -- the vehicle/content distinction just pretends like Occam doesn't exist and multiplies entities for the fun of it!
But of course all of this is commentary, "just those nerds arguing"
The purpose of this paper is to show up as an authoritative conclusion from a distinguished scientist at Deep Mind. And that's what it does.
Is the conclusion silly? OF course it is. Will it be quoted in the NYT? You Betcha!
> doesn't change the fact that it's software that requires human interaction to work.
Have you ever seen Claude Code launch a subagent? You've used it, right? You've seen it launch a subagent to do work? You understand that that is, in fact, Claude Code running itself, right?
I don't think subagents are representative of anything particularly interesting on the "agents can run themselves" front.
They're tool calls. Claude Code provides a tool that lets the model say effectively:
run_in_subagent("Figure out where JWTs are created and report back")
The current frontier models are all capable of "prompting themselves" in this way, but it's really just a parlor trick to help avoid burning more tokens in the top context window.
It's a really useful parlor trick, but I don't think it tells us anything profound.
The mechanism being simple is the interesting part. If one large complex goal can be split into subgoals and the subgoals completed without you, then you need a lot fewer humans to do a lot more work.
The OP says AI requires human interaction to work. This simply isn't true. You know yourself that as agents get more reliable you can delegate more to them, including having them launch more subagents, thereby getting more work done, with fewer and fewer humans. The unlock is the Task tool, but the power comes from the smarter and smarter models actually being able to delegate hierarchical tasks well!
Wtf? A sub-agent is a tool you give an agent and say "If you need to analyze logs delegate to the logs_viewer agent" so that the context window doesn't fill up with hundreds of thousands of tokens unnecessarily. In what universe do you live in where that mechanism somehow means you need fewer humans?
Do you think this means "Build a car" can be accomplished just because an LLM can send a prompt to another LLM who reports back a response?
Does your Linux server decide what processes it should launch at what time with a theory of what will happen next in order to complete a goal you specified in natural language? If so yes, I reckon you sure have!
Claude does not have a "theory" of anything, and I'd argue applying that mental model to LLM+Tools is a major reason why Claude can delete a production database.
Well, humans also routinely accidentely delete production databases. I think at this point arguing that LLMs are just clueless automatons that have no idea what they are doing is a losing battle.
They’re not clueless they just don’t have a memory and they don’t have judgement.
They create the illusion of being able to make decisions but they are always just following a simple template.They do not consider nuance, they cannot judge between two difficult options in a real sense.
Which is why they can delete prod databases and why they cannot do expert level work
Not sure if you are being pedantic but mathematics is quite different from other fields because it is highly structured, reasoning is explicit and it contains a dense volume of high level training data. Results are able to be verified easily due to its structure.
Even then, they are most effective in assisting and are not able to produce results independently. If you have proof otherwise I would love to read up on it
I like to think of LLMs as idiot savants. Exceptional at certain tasks, but might also eat the table cloth if you stop paying attention at the wrong time.
With humans, you can kind of interview/select for a more normalized distribution of outcomes, with outliers being less probable, but not impossible.
I mean maybe it’s a losing battle today, but it is correct. So in a few years when the dust settles, we’ll probably all be using LLMs as clueless automatons that still do useful work as tools
Maybe. But probably not. It doesn't matter if it's AGI though. If those other apps and tools do simple things that are predictable, then we can be pretty sure what will happen. If those tools can modify their own configuration and create new cron jobs, it becomes much harder to say anything about what will happen.
Most of us work on software that can modify its own configuration and create new jobs. I, too, have worked in ansible and terraform.
The key break here is the lack of predictability and I think it's important that we don't get too starry eyed and accept that that might be a weakness - not a strength.
My claude has never yet launched itself from my terminal, gave itself a prompt, and then got to work. It has only ever spawned a sub-agent after I had given it a prompt. It was inert until a human got involved.
If that is software running itself, then an if statement that spawns a process conditionally is running itself.
Substance aside, I feel this comment is combative enough to be considered unhelpful. Patronizing and talking down to others convinces no one and only serves as a temporary source of emotional catharsis and a less temporary source of reputational damage.
All AI requires steering as the results begin to decohere and self-enshittify over time.
AI in the hands of an expert operator is an exoskeleton. AI left alone is a stooge.
Nobody has built an all-AI operator capable of self-direction and choices superior to a human expert. When that happens, you'd better have your debts paid and bunker stocked.
We haven't seen any signs of this yet. I'm totally open to the idea of that happening in the short term (within 5 years), but I'm pessimistic it'll happen so quickly. It seems as though there are major missing pieces of the puzzle.
For now, AI is an exoskeleton. If you don't know how to pilot it, or if you turn the autopilot on and leave it alone, you're creating a mess.
This is still an AI maximalist perspective. One expert with AI tools can outperform multiple experts without AI assistance. It's just got a much longer time horizon on us being wholly replaced.
I remember this old thing called Bugs Everywhere -- it was a bug/issue tracker which actually lived inside your hg repository. I wonder if we could standardize on something like that? or git notes with an issues ref? or something magical like that?
Then it's BYOR -- bring your own renderer. Trivial CLI bugtrackers, agentic nonsense, pretty web stuff, whatever and the data lives in the repo.
And probably the network/black-hole effect of platforms like GitHub, Linkedin and the like are hard to achieve with fully distributed solutions, all the more when the other side is backed by huge capital which absolutely love concentration of power.
reply