I find it even harder to trust elected leaders from any party. At least Sam and Dario are aligned with a value set that is understood and clear, whereas political leaders values changes as do the polls their livelihood depends on changes.
> At least Sam and Dario are aligned with a value set that is understood and clear
What value set do you perceive that to be, and why would you take your perception of it to be any more sound than it would be with a politician?
It's not like someone can operate companies of that scale, especially startups, through earnestness and openness. Like national politics, their job is fundamentally about perception management and power brokering across dynamic windows of opportunity. Nothing they say or do can be taken at face value, and you can't reduce their incentives to either company or personal profit in any particular form over any particular time scale.
I feel like many of Anthropic's issues are due to Dario being too earnest and open. It seems both refreshing (that a CEO has thought deeply about and is willing to talk publicly about the dangers of their product) and depressing (that so many people cynically think this is some sort of marketing ploy).
I couldn't help but notice how each successive headline reporting our glorious victories seemed to draw closer to Tokyo.
Something like that.
Well. I can't help but notice how each successive headline reporting how this scam/stochastic parrot/"scare quotes intelligence" seems to be solving more and more things that were but a few years ago widely regarded as being indicators of high intelligence.
Year 3 of being told my job will be replaced by AI, and the only thing that's happened so far is that AI vendors keep showing up to my office, begging me to pay them to use it is a tool.
If you're only seeing the charts go up - you're not looking in the right places.
I'm looking at the millennium puzzles, and independently of those puzzles I had asked it for a fluid dynamics simulation engine that runs in my browser, and it put one together for me so I could play with aerospikes and watch the formation of Mach diamonds in rocket engines. The isochrone map generator has also been stuck on my to-do list for years, and yet now thanks to Claude, I have it, and it's real-time and multimodal.
Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write. Sure, limited speedup, Amdahl's law and coding is not the only part of the job, but anyone who was fine at PM and QA but not code no longer needs a coder.
Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.
I remember when the ability to generate music and art was "uniquely human", and sure there's a lot of cringe there with those models, but they're also winning awards and causing controversy by doing so, and artists are losing clients; I remember when the board game Go was considered to require "human intuition we could never make a computer solve, totally different to chess" (and I remember when chess was so, too).
When I was a kid, cheques and letters on addresses often got read by a human; the OCR which automated this is also AI, though these days image-to-numbers is the "hello world" of the field.
>Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.
okay, and?
>Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write.
Boy, programmers sure do think programming is like the only thing in the world
I'll repeat it for you again:
If you're only seeing the charts go up - you're not looking in the right places.
They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.
They may not work for your industry, but then again I don't know what your industry is.
Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's good at getting good. The most recent one being "make useful contributions to millennium prize maths problems". A year or two before that, it was even "do well on degree-level economics essays".
Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead. This is much smaller (strictly in terms of money) than with software, but is also measurable.
Now, if you said Tesla's self-driving cars (another AI) are a trillion dollar scam, that I would even agree with. At least, for the market cap part, for actual sales it's more like a billion dollar (ish) scam.
>They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.
Yeah, if you think I'm being overly literal. Regardless, people here were positing, yourself included, that they are widely useful.
> Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's
Again, over my lifetime, I've seen it been told to me that the next iteration of each model will surely be the one that does away with my entire profession. Yet, again, here we are, firms at my door, begging me to use their tools.
>Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead.
No, they are unhappy because it rips off their work. You're totally wrong about that.
It's exactly the opposite. Anthropic has earned their terrible reputation through years of lying, deceit, misdirection, gaslighting, unethical marketing strategies, etc.
I remember it vividly, when Anthropic first came on the scene, people (myself included) were incredibly optimistic about them and their leadership. Everyone hated SamA and OpenAI because they felt they couldn't be trusted.
Then slowly but surely, they showed their true colours. Now their reputation is in shambles due to their own behavior, and people are rooting for OAI to beat them. OAI's reputation gains have purely been a result of NOT following in the footsteps of Anthropic.
"Number go up" is table stakes, it's how you keep track of the score well before you reach their level.
However, that's like saying I'm motivated by food. I mean, yes, I like food, but this isn't a useful description to let you guess what move they'll make next, especially as they're opening opining about radical economic transformations that are likely to do to money what money did to real estate when the industrial revolution came.
And that's still true even if you don't believe they're anywhere near actually achieving any of these things.
I shouldn't be surprised, but it's still wild to me that people would trust robber barons rather than their elected officials to run their country. And this continues until you end up electing robber barrons as your elected officials, which is where we are at now.
The idea that anything of lasting good can come out such a preference doesn't seem conceivable. And if the educated think like this I guess we're passed blaming the poor and ignorant.
No system of governance can deal with immense concentration of power. The US Constitution was about separation of powers. Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful.
Political leaders become a problem when they amass too much power. Corporations become a problem when they amass too much power. It doesn't matter what Sam and Dario's purported values are. They aspire to power and absolutely power always corrupts absolutely.
Technologies which are infinitely powerful or whose power grows too quickly outrun any reasonable attempt at regulation. If you imagine that tomorrow everyone were given a tank, we might think, "alright, everyone has a tank so it's not too bad." But humans are squishy, and our houses are (relatively) squishy compared to tanks. Substantial collateral damage would result from everyone having a tank, and it seems likely that substantial collateral damage will result from everyone having a cyberterrorism-capable slop machine.
> Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful.
this is "direct democracy" and it's not even close to exist in USA... even with that a society can allow powerful people to exist if they don't create any law forbidding that
Democracy includes a broader array of governmental organization than just pure direct democracy. If you do believe that individuals should have some ability dictate the terms of their own social organization, then you believe in some amount of democratic principles.
Economic power eventually manifests in the political realm. The wealthy effectively get more votes, which means that society moves away from being democratic. Thus substantial wealth inequality is incompatible with democracy in the long run. We have been witnessing that corruption for a while now.
It's interesting the cyberpunk-esque future we're sliding into. Things like cognito-hazards and information-hazards are legitimately discussed and researched problems we're experiencing.
It's going to be interesting on how humanity deals with this problem (well, or if we turn it over to AI and make it their problem and suffer whatever consequences falls out). Being able to gather further information and power by acting on the information you already have causing massive power imbalances that is very hard to deal with, it's a natural outcome.
Funny, I'm sure I would have noticed when I visited if US cars came with tracks, a 105 mm main cannon, and massed around 55 metric tons.
We may all like our own personal R2 units, but if you insist on scifi, instead of tanks, consider everyone getting an X-wing for their commute. Oops, safety on the blaster was off, there goes the neighbourhood.
Cars are decidedly less dangerous than tanks, which are less dangerous than nuclear weapons. I am certain that giving a nuclear weapon to every person in the world would not go well.
I can't properly read the tank/car comment, the comparison is bizarre to me. But I think this misses the point a bit. The reality of these new risks is literally being learned in front of us in real time, and in my opinion, anyone who claims to understand these risks is speculating at best, and actively manipulating the situation for whatever reasons.
While I am a (mostly) capitalist and generally disagree with nationalization (including, for the time being, this situation), I also don't think we can say that "centralized power never works". Maybe we scope that a bit. I know plenty of business owners who centralize power in their businesses and they are effective, ethical and it works perfectly fine. In theory, in the US, nationalizing some unit of the economy _decentralizes_ power; the US is, after all, a representative democracy. Trustworthiness of the electorate is a different problem. But I don't think we can generalize much about centralization beyond sometimes it works and sometimes it doesn't. There is a difference between centralizing all power in a given administrative unit, and centralizing certain powers, but that's more analogous to non-democratic units.
Admittedly I haven't thought this through incredibly deeply but what if "nationalize" just means the US government owns half of the company? Then we get profits as recompense for building the company on our shared culture but there's still a profit motive for employees and a check on the direction of the company in the same way VCs have. But without necessarily turning the company into some red-tape bound bureaucracy.
> Then, in August, Trump called for Intel's beleaguered CEO Lip-Bu Tan to resign, alleging ties to China. Days later, after Tan met with Trump, the president called him a "success," before announcing that the federal government had bought a stake in the company.
Somehow, this isn't derided by the right as socialism.
> I find it even harder to trust elected leaders from any party.
You can vote out an elected leader, but not Sam and Dario. It's very weird that you're so willing to give up any kind of power and want to be ruled by unelected billionaires who only want to take advantage of you at every opportunity.
Yeah, the entire rest of the world has pretty much been stuck being ruled by unelected billionaires who only want to take advantage of them at every opportunity. It's a problem. It's starting to change though. The EU is starting to walk away from their abusive relationship with Microsoft and Amazon. China has a lot of their own stuff (mostly to better control their people though).
As an American, I'm still hoping it's not too late to fix things, but it's got to be hard for those outside the US to be so dependent on it, especially when we're looking like a sinking ship and our current administration is still running around drilling holes in the hull.
You can always become a US citizen to get a voice, but I wouldn't recommend it now or you'll be thrown in prison as soon as you show up to your scheduled immigration hearing. The better option is to keep trying to reduce your dependence on US companies and consider the worst aspects of our current situation (in both corporate policy and government) as a cautionary tale so you can try to avoid them in your own country.
Yes it's clear that Sam and Dario seem to be aligned with a value set that prioritizes concentrating trans-national government-mandated centralized control of AI and crowning themselves high priests of this unholy abomination. "At least it's clear that they're aiming to bring hell on earth" -- hard disagree, I think we can aim significantly higher.
> It can't be discoverable if it doesn't exist ...
This cuts both ways. I've seen plenty of litigation go south because one side had evidence and the other side had nothing because they deleted/shredded/lost the proof.
I'm fairly sure most "safety" people consider "large scale automated crime" part of the threat model, as the agents could accidentally fall into such a trap, if optimized for some misunderstood goal.
Or sue, or organize and protest, or win the next election. Not every government problem warrants revolution, and not needing to revolt is not a failing of the people.
> Americans/Westerners (left, right, center) are way too comfortable for a revolution
A better read is that the situation for most Americans and "Westerners" whatever that means, is that things are good enough that the people do not see value in revolution, or they are able to adjust the government's course enough through elections, redress, courts and other political feedback mechanisms.
> This is unsurprising to me, since people are often quite resistant to learning.
This is an attitude I had when I was 19 and had spent a bunch of time and money going to school. I had a life event that made me leave school and take on a huge debt that had to be paid. Most people learn - they just learn in context of their necessity and their own passions. For other things, they lean on the community around them (doctors, lawyers, accountants, software engineers etc...)
> AI seems to be a good tool for people who are already good at learning and critical thinking to use to accelerate their learning.
I'm watching the huddled masses start to figure it out. AI is going to be to arrogant intellectualism what the gun was to the knight. Now anyone who can operate a mobile phone (and literacy is not required) has access to a shocking level of expertise and contextualized knowledge. And as AI improves this will be more and more the case.
> constantly missing words due to a lacking vocabulary
First prompts of non developer employees were "what does ___ mean?". Real time vocabulary is a thing now.
> “well, maybe literacy is actually not so important,” is so utterly ridiculous that I hope it’s satire and I’m just missing the joke.
It's not a joke. AI is going to reshape society. It's just started. Back to the gun analogy - the knights are just now starting to realize that proficiency with a sword and 70kg of armor are no match for a peasant with a pistol. Likewise, I suspect that Joe six-pack with AI in his pocket can do knowledge work at a "way good enough" level. I don't want to accept it, but that seems to be reality.
> - the knights are just now starting to realize that proficiency with a sword and 70kg of armor are no match for a peasant with a pistol
This isn’t a metaphor/analogy that works. There is a parallel in the introduction of a tool that has more “firepower” but that isn’t the crux of the issue when it comes to intelligence.
You can give fable to someone who has never written code before and someone who has experience writing SW. Your outcomes are very different.
The crux of the matter is that you need to know what “good” looks like to use automation or augmentation tools well. This is why it’s a greater boost to people with experience.
> It kind of seems like an insane TLD structure to begin with, right?
It's been around for years. I seem to remember this issue coming up around 2001 where originally .name was for third level registration (i.e. john.doe.name) and changed to second level it a few years later and caused some problems... https://publicsuffix.org/ talks about it in light of architectural limitations of domain names.
> can Joe set a cookie on all of .smith.name?
That can happen. I seem to remember ancient browsers made it so .name (and other non-generic TLDs) required three periods. I think country code domains and new generic TLDS caused the browsers to change it.
It's pretty screwed up, but a lot of the people with .name domains have had them for a very long time. Sad to see them all lose their identity online that way.
reply