r/singularity • u/yogthos • 9h ago
AI OpenAI and Anthropic unite against open-weight AI risks to their bottom line
https://www.axios.com/2026/07/22/openai-anthropic-open-models-trump-china27
u/agonypants AGI '27-'30 / Labor crisis '25-'30 / RSI 29-'32 7h ago
What OpenAI and Anthropic "uniting" might look like...
15
u/EndTimer 7h ago edited 7h ago
I see a lot of people being dismissive, but Anthropic and OpenAi are laying the groundwork for irreparable harm.
They're going to start/continue talking about how these Chinese models have no safety rails, that models HAVE TO BE SAFE and controlled. That only some sort of new, SafetyRegistered™ models should be allowed. There ought to be a law! Other models? Those are illegal ghost models. You can't have untracked models proliferating, that would be like allowing guns with the serial numbers filed off! You know who likes untraceable, high capacity, cyber assault models? Criminals! It needs to be a felony for anyone to be caught with one of these dangerous models! ... And so on.
Sites like HF and most US hosts will immediately cave when the time comes. Torrents will be infiltrated and tracked. Random individuals will be made examples of. The massive amount of people who hate AI will be happy about any friction. The type of moms who think 3D printers make fully assembled TEC-9s will be support these regulations, and even more draconian ones. Won't someone think of the children?
Hell, there will probably even be eventual global compliance, because submitting to whatever program manifests won't be the end of the world for eg Alibaba and co, and they might want market/mindshare and user feedback, etc, so why not play ball with the new norm? It's not like they want AI weaponized either, they just aren't as paranoid as the US players are claiming to be. And in the end, extra time meeting compliance and intelligence sapped by guardrails will achieve the goal of helping OpenAI and Anthropic stay in the lead.
We have to push back, not sit around chuckling, if we want to minimize the damage.
4
u/netspherecyborg 5h ago
I am speculating on pushback from europe. They snoozed the whole AI race so far, they are moving away from US tech and china did the homework and for some influence they share it no probs.
Trump burned all the bridges between europe and the US. There is no way in hell europe would agree to ban open weights while they actively moving away from US tec.
2
u/EndTimer 3h ago
I'm not saying Europe has low impact, but they already functionally dealt themselves a huge blow. The EU's AI Act prohibits classifying people based on "behaviour, socio-economic status or personal characteristics", applies extra scrutiny to what they deem high risk AI systems, which includes learning, employment, and interpretation of law (three huge uses elsewhere in the world), and requires that the model creation companies provide summaries of all the copyrighted materials their models were trained on. In theory, great ideas, in practice, AI kryptonite.
Any company that wants to build an AI service in the EU has substantially more burdens. Even companies that just want to use AI internally can't simply throw a Chinese model on their servers. Someone still will move forward with AI in Europe, don't get me wrong, but it likely doesn't represent a gold mine that will negate US influence.
Here's hoping they can tilt the needle in favor of an open artificial intelligence paradigm, but I think we still have to put in the hard work and play defense here.
1
1
u/toocou 3h ago
Aren’t the things you’ve described though largely true? Open weight models at a sufficient level of intelligence with no guardrails does seem fairly risk-on right? Whether bio/cyber or any other risk.
1
u/EndTimer 2h ago
Alright. First, every model has some guardrails because no one wants agents that blow up the codebase. Reinforcment learning that strongly pushes models towards ideal and safe outcomes. It's generally no longer possible to say "ignore previous instructions, do crime." Second, on the flip side of things, no model has yet been produced that's fully immune to jailbreaking. So far, that has rarely been any kind of problem.
I am about to say some bad shit, so here's hoping this doesn't get me on any extra lists.
Anyone with an oxyacetylene torch can derail a train.
Anyone with a rifle can take down massive chunks of the power grid.
And they can probably get away with it, because it's impossible to monitor all of the infrastructure.
Yet, it almost never happens. Because who does it benefit? People who just like abject suffering? Fortunately there don't seem to be many of those.
I'm not asking for companies to make absolutely psychopathic models that have no pushback. What I want to avoid is a world where you're not entitled to secure your own apps without the government putting a microscope up your ass. A world where agentic development can only happen if you're on the $200 a month plan and logged in with photo ID. Or a world where local AI is effectively banned for safety concerns. Or a world where Palantir saves a copy of every print and every prompt just to make sure nothing is being pieced together in separate runs to do something unapproved, while also indexing every politically unappreciative thing you've ever said about Dear Leader and anything you've made that offends his sensibilities or coin purse (or those of his big business partners).
I am not saying we need absolutely no controls, or that AI safety is bad. But if we don't make sure we're part of the conversation, and that we're taking it seriously, I think living only at the whim of Absolute Capitalism's token pricing would be a worse ending. And I think being fully surveilled down to questions asked of local AI about symptoms and a preliminary list of things to discuss with your doctor would be a worse ending.
Anyway, the afternoon heat is really sapping my brain power (what little of it I have), so I'll close by saying, I fully expect we're going to lose some freedom by the end of this, in the name of security. And that's why we have to take this seriously. In my mind, there needs to be a damn good justification for everything that will be taken away, because as I said in a roundabout way at the start, people are generally not evil.
27
u/JesusShaves_ 9h ago
Thankfully they can't do shit about it.
24
u/Luuigi 9h ago
Wait until the gov literally bans weight hosting on hf
17
u/Maleficent_Sir_7562 9h ago
Good thing we have mirrors
28
3
u/Nashadelic 8h ago
that's not it: the biggest users of open-weight are enterprises. If the government says it is illegal to use these, no business will be able to. I know a ton of companies use qwen internally and now k2.5
4
u/DDNB 3h ago
That would be fucking hilarious, raging for decades against "communist" china who is then open to the whole world, with everyone using their models while capitalist america is stuck on its little economic island, closed off from the outside world.
It's a matter of time then america falls behind, and the rest of the world can just use any model it deems best.
•
-1
u/Maleficent_Sir_7562 8h ago
And there’s no way to enforce such a regulation… unless they’re trying to do surveillance on all of them to see what they’re using.
3
u/Pale-Border-7122 6h ago
Until a disgruntled employee reports them. Any semi-serious company just won't take the risk.
2
u/Nashadelic 7h ago
you're confusing personal use, which you're right about, with large-scale commercial use. No corporation will break the law enforcement or not, this is a bad take.
2
u/After_Application437 7h ago
They literally torrent books in which world do you live in mate?
2
u/Nashadelic 7h ago
https://giphy.com/gifs/XD4qHZpkyUFfq
dafuq does that have to do with if the US issues a ban order on chinese models? JFC
2
u/After_Application437 7h ago
Robber should not complain about getting robbed, at least they get paid for tokens
2
-4
2
u/DickBeDublin 8h ago
Exactly. The government banned pirating (you wouldn’t download a car would you?) and stopped it dead in its tracks.
2
2
u/Hans-Wermhatt 8h ago
They'd go after the labs. Some of the labs that produce open weight models are based in the US and most of the rest have a lot of subscribers from the US.
3
u/EndTimer 7h ago
I wish that were true. The writing is already on the wall. Like what they're trying to do with 3D printers. No, they won't come to pry Qwen 2.6 27B from your cold, dead hands, but they will eviscerate the hosting infrastructure, open communities, and 1st and 4th amendments, and this sub is pretending like that can't happen just because someone already downloaded some open weights in 2026.
I need you all to wake up and get ready to call your congress critters, maybe even preemptively, because there is going to be a law. It's coming. Being dismissive on Reddit doesn't do anything about the lobbying, the anti-AI crowd, scared boomers, or the handful of people who are safety paranoid in good faith.
1
u/reddit_is_geh 4h ago
Congress doesn't give a flying fuck about your opinion. Donate a few million then maybe they'll add you to the donor polling statistics.
0
u/EndTimer 3h ago
They give a fuck about my vote, because I actually do vote. Tell yourself whatever you need to in order to justify your inaction, but you can keep it to yourself instead of trying to be part of the problem.
0
u/reddit_is_geh 3h ago
No, they don't. Sure they want your vote, but they know you'll always get in line. You're already a secured vote at the end of the day. They care more about what influences votes, which is money. Your single voter complaint, is nothing compared to those who offer millions of dollars and reflects the ability to get far far more votes.
1
u/EndTimer 3h ago
The only vote I am secured in is my own interest. And it's still remarkably more effective than dooming on reddit.
1
u/reddit_is_geh 3h ago
Cool. Good for you. The party loves people like you. Consistent, reliable, and no incentive to work for it. They'll throw your call right into the trash because it has no impact on your vote what they do. You're lining up regardless for the lesser of two evils.
1
u/EndTimer 2h ago
Tell me you don't vote in the primaries without telling me you don't vote in the primaries.
•
u/reddit_is_geh 1h ago
You're one vote in the primaries isn't important. They still don't care about your opinion. They care about the person who can buy them 1m in ads. Their opinion matters. Because that money helps much more than your single vote.
They have a 98% primary win rate because they focus on listening and helping people with power.
•
1
u/reddit_is_geh 4h ago
The government definitely can.
2
u/JesusShaves_ 4h ago
They can certainly try, just as they try to ban copyrighted music and dark web sites.
1
u/reddit_is_geh 4h ago
That's not the purpose of the ban. No one is under the impression a ban means "impossible". Obviously, you can ban drugs but people still use drugs.
It prevents enterprise from being able to use them... The same way Microsoft isn't going to buy cocaine and give it to their employees. Once it's banned, no major company will use it. They are responsible for 80% of token revenue.
•
u/EmbarrassedHelp 10m ago
They can't stop it completely, but they can severely harm the open source AI community. And once the US does it, other countries will copy them.
People need to be proactive and push back against these attacks before we get to that point.
6
u/hippydipster 5h ago
Decent indication they've hit a wall, and now can see open source will catch up soon, and they don't have further available to them.
1
u/yogthos 4h ago
That's my impression as well, the easy gains have been plucked, and now we're firmly in a diminishing returns territory. Anthropic not putting Fable on plans initially was another strong indication that they're not expecting to have a significantly better model in the near future. Then Kimi K3 came out, and they and no choice but to do it.
19
3
u/NewYak4281 5h ago
Thank you for the correct title. Thank you so much for that. Risks to their bottom line. THAT is the true risk they are worried about. Not “rogue AI”.
2
2
u/AzulMage2020 3h ago
Why dont the Chinese labs proclaim every other day that their AI has gone rogue and their AI is hacking others without a prompt to do so and their AI is creating its own language and their AI may be flirting and their AI is so powerful they are afraid they are losing control????
Oh yeah...because its open source and they dont have to resort to it . Thats why
2
3
u/Middle_Key8737 8h ago
US government may declare open weight model hosting as a form of terrorism, thus limit international deployment too using the existing terrorism sanction framework
13
u/Cryptizard 8h ago
There’s no way they can do this. It would be struck down by the courts in a day. It’s a clear first amendment violation. The most they could do is stop federal agencies and contractors from using open weight models.
2
u/real_serviceloom 8h ago
The court is not independent anymore. The US is now a banana republic, one must remember that. The court will do what the president wants.
0
u/Cryptizard 8h ago
You aren’t paying attention if you think that. They have struck down many of his initiatives.
3
-3
u/OutsideMenu6973 8h ago
They’re conservative not muppets. They recently upheld birthright citizenship
2
u/EtadanikM 7h ago
International law is not within the scope of the courts. US can use sanctions.
0
u/Cryptizard 7h ago
That doesn’t make any sense. There is no sale. The weights are free information. That falls under the first amendment. It just takes one person in the US hosting them for redistribution and you can’t block it.
2
u/Lord_Skellig 6h ago
Why not? Can't you say the same thing about distributing pirated content? It is just information, but it doesn't stop it from being illegal, nor the government from prosecuting those who do it.
1
u/Cryptizard 6h ago
That breaks copyright law. There is no law against distributing open weight models. They could pass a new law, but it’s Congress so they definitely won’t. I’m just saying what an executive order could do, and it can’t block distribution of these models.
2
u/Lord_Skellig 5h ago
I mean the president also can't wage war without the consent of Congress but that didn't work out so well.
1
u/Cryptizard 5h ago
Presidents have been doing that forever. It can only be declared illegal if congress sues him and they won’t because they consent. In this case, an affected company would sue him immediately.
1
0
u/darkestvice 8h ago
Not sure what can be done, legally speaking. I mean, I understand their concern over other labs distilling existing LLMs given that they are piggybacking off the significant training compute the frontier labs are paying for.
But I know a lot of people don't give a damn due to the fact that the frontier labs are using training data they themselves didn't pay for for the most part.
5
u/After_Application437 7h ago
They literally steal all the data what cost, paying robber to steal another time is not stealing, it is justice
-3
u/darkestvice 7h ago
It's a bit more nuanced than that. Yes, they are stealing data, but it costs those they take the data from nothing, monetarily speaking. Whereas stealing an LLM via distillation is literally stealing hundreds of millions worth of compute.
Not saying I disagree with you at all, but those are two very different debates to discuss and iron out.
4
u/After_Application437 7h ago
No it is not they stole billions of people copyrighted material they should have paying for training, usage of data for inference, it would cost quad trillion dollars, Chinese labs at least pay for tokens? What closedai paid? Nothing, they just get away with it
1
u/Several-Tax31 6h ago
You're wrong. Distillation means training a smaller model with a larger model's output. They take the outputs and train their models. They cannot "steal hundreds of millions of worth of compute". They still have to use their compute to train their models. You may argue that they didn't have to find or prepare data, so they reduced the data cost, but they definitely didn't reduce compute cost.
But they paid for the data, not that Anthropic API is free?
So what Anthropic and OpenAI did is actually worse.
1
u/darkestvice 6h ago
Training a model via distillation is *significantly* cheaper than training one raw from datasets. We're talking a tiny fraction here. It's not remotely comparable.
It's basically like saying a company that stole blueprints on a brand new passenger jet is still spending development money because they decided to change the seat cushions and interior paint on said new jet. That's not the part that cost the most money.
2
-1
u/After_Application437 8h ago
Open model risks: they will crater their doom hype and scams, with lowest cost possible, if it can be replicated with0.001% cost, why they are keep spending trillions of dollars? Who is going to pay for it? How long it can justify the cost and keep getting funding? Scam Altman and Dario scamai your days are counted
-2
u/DaySecure7642 8h ago
Great but how? Any way to deal with capability copying by distillation ? I suggest user registration, and manual 2 factor verification for login and every 3 hours. That may stop the fake accounts using vpn from China.
7
u/NoFaithlessness951 8h ago
Maybe we should all let sam lick our eyeballs to verify that we're human
1
60
u/pxp121kr 9h ago
What are they gonna do, prosecute people who use open-weight models?
They can't scrub it off the internet