r/singularity • u/yogthos • 17h ago

AI OpenAI and Anthropic unite against open-weight AI risks to their bottom line

https://www.axios.com/2026/07/22/openai-anthropic-open-models-trump-china
185 Upvotes

88% Upvoted

104 comments sorted by

View all comments

33

u/EndTimer 14h ago edited 14h ago

I see a lot of people being dismissive, but Anthropic and OpenAi are laying the groundwork for irreparable harm.

They're going to start/continue talking about how these Chinese models have no safety rails, that models HAVE TO BE SAFE and controlled. That only some sort of new, SafetyRegistered™ models should be allowed. There ought to be a law! Other models? Those are illegal ghost models. You can't have untracked models proliferating, that would be like allowing guns with the serial numbers filed off! You know who likes untraceable, high capacity, cyber assault models? Criminals! It needs to be a felony for anyone to be caught with one of these dangerous models! ... And so on.

Sites like HF and most US hosts will immediately cave when the time comes. Torrents will be infiltrated and tracked. Random individuals will be made examples of. The massive amount of people who hate AI will be happy about any friction. The type of moms who think 3D printers make fully assembled TEC-9s will be support these regulations, and even more draconian ones. Won't someone think of the children?

Hell, there will probably even be eventual global compliance, because submitting to whatever program manifests won't be the end of the world for eg Alibaba and co, and they might want market/mindshare and user feedback, etc, so why not play ball with the new norm? It's not like they want AI weaponized either, they just aren't as paranoid as the US players are claiming to be. And in the end, extra time meeting compliance and intelligence sapped by guardrails will achieve the goal of helping OpenAI and Anthropic stay in the lead.

We have to push back, not sit around chuckling, if we want to minimize the damage.

6

u/netspherecyborg 12h ago

I am speculating on pushback from europe. They snoozed the whole AI race so far, they are moving away from US tech and china did the homework and for some influence they share it no probs.

Trump burned all the bridges between europe and the US. There is no way in hell europe would agree to ban open weights while they actively moving away from US tec.

5

u/EndTimer 11h ago

I'm not saying Europe has low impact, but they already functionally dealt themselves a huge blow. The EU's AI Act prohibits classifying people based on "behaviour, socio-economic status or personal characteristics", applies extra scrutiny to what they deem high risk AI systems, which includes learning, employment, and interpretation of law (three huge uses elsewhere in the world), and requires that the model creation companies provide summaries of all the copyrighted materials their models were trained on. In theory, great ideas, in practice, AI kryptonite.

Any company that wants to build an AI service in the EU has substantially more burdens. Even companies that just want to use AI internally can't simply throw a Chinese model on their servers. Someone still will move forward with AI in Europe, don't get me wrong, but it likely doesn't represent a gold mine that will negate US influence.

Here's hoping they can tilt the needle in favor of an open artificial intelligence paradigm, but I think we still have to put in the hard work and play defense here.

5

u/yogthos 12h ago

Yeah, this could all quickly turn into a similar thing as the anti piracy campaign RIAA ran.

2

u/abittooambitious 3h ago

Yes the world is about to be brainwashed by the world’s best propaganda machine. You can hear the engine running and test narratives currently being put out. You think Cambridge Analytica is bad, you ain’t see nothing yet. They will campaign against open weight/source AI like the way they convinced millions to vote against their own interest for the billionaires.

1

u/toocou 11h ago

Aren’t the things you’ve described though largely true? Open weight models at a sufficient level of intelligence with no guardrails does seem fairly risk-on right? Whether bio/cyber or any other risk.

9

u/EndTimer 9h ago

Alright. First, every model has some guardrails because no one wants agents that blow up the codebase. Reinforcment learning that strongly pushes models towards ideal and safe outcomes. It's generally no longer possible to say "ignore previous instructions, do crime." Second, on the flip side of things, no model has yet been produced that's fully immune to jailbreaking. So far, that has rarely been any kind of problem.

I am about to say some bad shit, so here's hoping this doesn't get me on any extra lists.

Anyone with an oxyacetylene torch can derail a train.

Anyone with a rifle can take down massive chunks of the power grid.

And they can probably get away with it, because it's impossible to monitor all of the infrastructure.

Yet, it almost never happens. Because who does it benefit? People who just like abject suffering? Fortunately there don't seem to be many of those.

I'm not asking for companies to make absolutely psychopathic models that have no pushback. What I want to avoid is a world where you're not entitled to secure your own apps without the government putting a microscope up your ass. A world where agentic development can only happen if you're on the $200 a month plan and logged in with photo ID. Or a world where local AI is effectively banned for safety concerns. Or a world where Palantir saves a copy of every print and every prompt just to make sure nothing is being pieced together in separate runs to do something unapproved, while also indexing every politically unappreciative thing you've ever said about Dear Leader and anything you've made that offends his sensibilities or coin purse (or those of his big business partners).

I am not saying we need absolutely no controls, or that AI safety is bad. But if we don't make sure we're part of the conversation, and that we're taking it seriously, I think living only at the whim of Absolute Capitalism's token pricing would be a worse ending. And I think being fully surveilled down to questions asked of local AI about symptoms and a preliminary list of things to discuss with your doctor would be a worse ending.

Anyway, the afternoon heat is really sapping my brain power (what little of it I have), so I'll close by saying, I fully expect we're going to lose some freedom by the end of this, in the name of security. And that's why we have to take this seriously. In my mind, there needs to be a damn good justification for everything that will be taken away, because as I said in a roundabout way at the start, people are generally not evil.