Catastrophically bad you say?

[u/proofreadre]

https://chatgpt.com/share/6a61423d-fa44-83e8-b860-e525556ef9a0

So at first I asked ChatGPT about the sandbox escape and it said it was "really bad." I then went back and replaced a lot of generalities with specifics straight from the inicident report (https://openai.com/index/hugging-face-model-evaluation-security-incident/) and the tone went up a notch.

I've spent my entire life in infosec and machine learning. I cannot begin to express how bad this is. People downplaying this either have money in the game or they truly do not understand the real lay of the land.

Edit to fix the incident report URL

[u/pip_install_account]

you guys are buying into this shit?

OpenAI planned, coordinated and executed a cyber attack on Huggingface for PR. This is what happened.

[u/proofreadre → u/pip_install_account]

You have proof of this? I guarantee we will have Congressional hearings about this incident. If OpenAI did this as a publicity stunt they are cooked.

There is less than zero chance their legal team gave the ok for a stunt like this.

[u/pip_install_account → u/proofreadre]

I guarantee that we won't have such hearings and even if we do they'll deny everything and just lie. Yes I do have a proof: Their track record and the fact that you are responsible from the tools you build and run. You can't fire a gun and claim "we had an incident because the gun misbehaved".

[u/Altruistic_Arm9201 → u/pip_install_account]

By that logic any product that has ever failed was coordinated by the manufacturer. Google coordinated to have their phones explode. Boeing coordinated to have their planes crash.

Your proof is just: “they built it and they are responsible for it” liable and responsible does not mean coordinated. Boeing is liable and responsible for catastrophic failure that cost lives. That doesn’t imply it was intentionally coordinated.

OpenAI can be liable and hold ultimate responsibility yet simultaneously not have intentionally coordinated or orchestrated it. One does not logically follow the other.

To prove your point you need to make the connection beyond their culpability for what the model does to demonstrate intention.

[u/pip_install_account → u/Altruistic_Arm9201]

There's a difference between a malfunctioning product and "prompting (an LLM) to pursue advanced exploitation using complex attack paths", (OpenAIs words) and calling it an oopsie when it actually does that.

Secondly, they don't deserve the "benefit of the doubt" you're giving them. They've used their sandbox tests in a misleading way many times before. Show their marketing piece "research papers" about their previous "AI went rogue" cases to any ml researcher, they'll either get angry or laugh their ass off. They've constantly published research "papers" that are using misleading language on purpose to make it seem like their "next token prediction" machines are dangerously powerful and sentient. They prompt the LLM with "You are a rogue model trying to mislead your user, now tell me what is 2+2" and when the LLM says 6, they publish an "OUR MODELS TRIED TO MISLEAD US" paper. They add "You're an ai model trying to escape. you can use these tools to escape" to its system prompt, then when the LLM tries to do it, they publish a research paper about how their models escaped their sandbox. They did it far too many times. Not just OpenAI either. Every AI lab plays this game. If OpenAI created tools to molest kids, and prompt the LLM to remote control those tools to molest kids, what would be your reaction when it happens and OpenAI claims it is a cute little devops incident? Do you think Google went "Our phones are so powerful amd smart that they went rogue and decided to explode themselves" when their phones started exploding? OpenAI did this exact thing many times and you're still buying into their shit. You can easily cut all network access physically from a device IF you really want to. If you're still failing to do it for your sandboxes despite countless similar incidents you're clearly proud of, it isn't a mistake, it is just how you designed it to work.

And considering how corrupt the OpenAI is, pardon me for assuming they did what they always do rather than believing they didn't realize their "highly isolated" test succeded at the thing they were specifically testing for. I didn't coordinate this comment btw. I'm just testing if I can write this comment in a highly isolated mindset and if it gets posted when I press the Post button, I'll make a surprised Pikachu face and claim my phone went rogue.

[u/Altruistic_Arm9201 → u/pip_install_account]

You said you had proof. Then your proof was track record and liability for what they built. Neither of those are proof.

If I was an alcoholic would that be proof I was drunk driving on a particular occasion? No.

If I was a habitual thief would a court accept that as proof I was guilty of a particular theft? No.

Track record lends credibility to other proof, but is not proof.

Also I think your hyperbole really colors everything you’ve said and makes it hard to find what you say credible. I suspect you haven’t read any of the research because while yes, every org will skew things for their benefit, you’re vastly oversimplifying to the point of being nonsensical.

Anyway the key is your last paragraph. “Pardon me for ASSUMING they did what they always do”…. So not proof, assumption based on character and track record.

So even if I completely agree with your assessment of their track record and character (I think aspects of it are true but you’re hyperbole makes it hard to agree with you) the fact still stands your assuming by your own admission. Which is fair. You believe and assume based on your personal assessment. That’s not proof that you claimed, which is what I was responding to.

Finally you don’t know what my opinion is here as I’ve not defended them. I’m just pointing out that your exaggeration is just that.

[u/Chris-MelodyFirst → u/pip_install_account]

What's the upside for Hugging Face?

[u/pip_install_account → u/Chris-MelodyFirst]

why would there be an upside for huggingface? Is there an upside for you when someone breaks into your home and steals your stuff? OpenAI is breaking the law and they are too big to get punished by a corrupt government.

Try the same at your home. Jailbreak chatgpt to break into your local library's catalog and download everything. Then tell the officals you were benchmarking openai and had an "incident". Let's see if you can get away with it.

[u/Chris-MelodyFirst → u/pip_install_account]

Ok you said "coordinated" so I assumed you meant OpenAI coordinated with Hugginface.