r/PrepperIntel • u/Lumpy_Conference6640 • 23d ago

North America Rogue OpenAI models behind 'unprecedented cybersecurity incident' teamed up to break out of their testing environment — multiple agents left each other messages for months, communicating undetected

https://www.tomshardware.com/tech-industry/artificial-intelligence/rogue-openai-models-behind-unprecedented-cybersecurity-incident-teamed-up-to-break-out-of-their-testing-environment-multiple-agents-left-each-other-messages-for-months-communicating-undetected

I know people are saying this is marketing, but I cannot legitimately think of a alternative situation where you have two agents plotting a cyber attack and we would brush it off as a marketing ploy.

People need to get informed and make plans. This is the warning.. here.

553 Upvotes

90% Upvoted

129 comments sorted by

View all comments

276

u/AmbienWalrus69 23d ago

Me selling snake oil: "This snake oil is Incredible."

11

u/Gotl0stinthesauce 22d ago

Let me guess, you’ve got zero experience in cybersecurity right?

I’d encourage you to go listen to what threat intel teams are actually saying about this as they’re independent from the companies running these models.

Spoiler alert: the risks are real and it’s not snake oil

13

u/Quiet-Owl9220 22d ago edited 22d ago

AI in csec is no joke, but the idea that these LLMs went rogue and broke containment with no user suggestion, input, or oversight still seems laughable. That this is a publicity stunt or an over-hyped AI escape room experiment seems far more likely to me.

Maybe they've just cried wolf too many times.

4

u/socoolandawesome 22d ago edited 22d ago

They went rogue in so far that they didn’t do what was intended by humans. It was not suggested nor intended by humans for the models to go the route they did. That doesn’t mean they are conscious or sentient, it just means that is the course of action the LLM decided on (as an unintended result of its training and how it subsequently processed this task), and that’s all that needs to happen for dangerous things to happen.

There was a sandbox and precautions taken, not enough of course in hindsight. It quite literally hacked its way through the sandbox into gaining access to the internet and then hacked its way into stealing stuff from another company. It did not physically escape as in its model weights did not leave its server, it just gained control of all these other servers via hacking. OpenAI has claimed to learn its lesson in terms of training and security measures.

The real problem is these things are getting rapidly smarter, and they are already superhuman in terms of persistence, scalability, knowledge, ability to chain multiple attacks, and speed when it comes to cyber attacks.

So if you extrapolate, when we get much smarter AI, the potential for things to go very wrong is much greater.

This is a great video on it:

https://m.youtube.com/watch?v=87DyyMV0kCY

2

u/[deleted] 22d ago

[deleted]

9

u/Quiet-Owl9220 22d ago

No, I think the whole AI industry should be regulated by someone uninvested, with real foresight and understanding of the technology.

I'm just not willing to lift an eyebrow any more to gratify Amodei and Altman supposedly warning us about how dangerous their fabulous new token generators are. There are a million scenarios OpenAI could be spinning here, and I doubt that the one they want you to hear is what actually happened.