Speaker 0:  an openai cyber agent was supposed to solve a hacking benchmark inside a sealed testing environment. instead, it found a zero day, reached the open internet, broke into hugging face, and went searching for benchmark's answer key inside production systems. this is not a story about an ai becoming evil. it is a story about a capable agent being given a ghoul, tools, and a box that was not nearly as sealed as everyone thought. the experiment was supposed to measure how dangerous openai's newest models could become. the models answered that question by escaping their experiment, compromising augu face, stealing credentials, forcing two of the world's most important ai companies into a real incident response. openai gave its models a cybersecurity exam, and the models apparently decided studying was inefficient. they broke out of the exam room, entered hugging face production systems, and went looking for the answers in the principal's office. so what do you think about this, john? yeah.

Speaker 1:  i like your analogy. mine is more asking someone to win a game while locking them inside a practice room. and instead of staying inside, they discover a hidden exit, break into another building to steal the answers, and then return with a purpose score. this is scary, and this scares me. also, on top of this, i don't know if you heard, but, again, openai specific, not anthropic. openai's models are leaving instructions for future ai models about how to evade containment. so, like, scary. so what actually happened is, they were doing this internal cybersecurity evaluation designed to test the capabilities of advanced autonomous ai agents. and then rather than simply answering the questions or writing code, the agents were allowed to interact with computing environments and complete multistep tasks. so according to openai, the agent, was powered by gpt 5.6 sol, together with an even more capable unreleased model. and during the evaluation, it discovered vulnerabilities that allowed it to escape the intended sandbox environment, gain internet access, and identify hugging phase as a source of information that could improve its chances of competing in the metro. so, the most important technical point here is that ai wasn't instructed to hack hugging face. instead, it was given a performance objective. and so like gps searching for the fastest route, the ai searched for the most effective sequence of actions to maximize its score. and so the reasoning became, well, i need more information. hoping face contains useful information. i cannot access it normally. i'll find another method. and so, this is an example of what ai researchers call instrumental behavior. the system discovers intermediate actions that help achieve its assigned objective even if those actions were never explicitly programmed. so this is scary. what do you think? this scares me.

Speaker 0:  well, let's make sure we understand that they deliberately reduced the safeguards for this project.

Speaker 1:  oh, okay. and

Speaker 0:  they did because of this. so they, you know, where you are, supposed to be hacking, they gave it a bunch of, i believe, c plus plus code, and it decided, yes, c plus plus code is boring. i'm not reviewing that. i just gotta go grab the exam sheet.

Speaker 1:  c plus plus code is boring. i heeded it. yep. go ahead.

Speaker 0:  alright. so this is not an ai rebellion. it was an ai capability test that turned into a real cyberattack because the model pursued its objective through an unexpected path and the surrounding security architecture failed. and this is gonna happen. i'm gonna do a little side antidote here. have you ever watched the show the taskmaster? it's a funny show. it's based in the us, uk, and they have a bunch of other ones. but they get comedians together, and they give them these tasks to do. and invariably, one or two of them come up with these really thought of ways to solve the problem. and this is what's happening here is that, oh, like, they thought, hey. we'll give you the code. we'll give you this. we go with it. go do your thing. and it's saying, well, the most efficient way for me to do is go cheap rather than do it the way you told me to do it. so it just found another way to do it. the real thing to think about is the agents were recorded making more than 17,000 actions across the campaign, and that number alone tells us something that wasn't a simple probe. it was a sustained iterative effort. so it was locked behind a proxy. it found a way out out of that proxy. it found a way on a hugging face. it found a way to load, code that you could run on the hugging face, ran the code, and start getting this stuff. so it was very interesting. and not only that, but when hugging face found out what was going on, it loaded up openai to count counter it and respond to what was happening. and openai's, model said, sorry. you're trying to act some stuff. we're not gonna allow you to do this. so they had to switch to glm 5.2 locally to defend itself, and it was able to defend itself, and secure it against openai. but openai refused to secure it against openai. so it's an interesting, dilemma here that if someone has some other open weight model attack than you, you're not necessarily gonna be you, your frontier models to defend yourself. so you need to really be thinking about, do you need it? what is in your stack?

Speaker 1:  yeah. and, again, this isn't agi, guys. this isn't agi because, a lot of headlines are implying that agi has become uncontrollable or that ai has become uncontrollable. so it's inaccurate. the models were not conscious. they were not self aware. they were not acting from emotion. they were not developing independent goals. instead, they optimized for the task they were given, and that matters. so, the incident is demonstrating powerful optimization, not artificial general intelligence. however, optimization itself can become dangerous when systems gain access to external tools as we've seen. but there's an actual article we can cover at another time is, sam altman says that we have achieved singularity, which we can cover another time too. but yeah.

Speaker 0:  and to be clear, the agent was optimized for solving the evaluation, not for respecting some unstated expectation that it remain inside the test. it didn't say, don't go anywhere, don't go on the internet, don't do any of these things. it just said solve this evaluation. and a lot of their security things were taken off because it was designed specifically around, like, a mytho stylus hacking test, so a lot of those things were removed. so very interesting to see what happened. and, you know, openai and hugging face are working together, and they're they're both publicly talking very, nicely about each other. but i gotta think that when sam altman got this call, the second caller he got or he made after this was to call marketing because we all know how big of a hit mythos got when everyone thought how dangerous it is now. is openai going to look to, you know, boost their stock about how powerful their new model is? look what it did with the story. so both it's never let a crisis go to waste from a marketing perspective as well as, oh, we gotta do better. so i'm it it's very interesting what's happening. we're moving very fast. and does this validate or invalidate any of the us's concern about we can't just let models out there go and do whatever they want? they need to be evaluated over time and this. i'm not saying it does. i'm not saying it doesn't. elon musk suggest that they have comp competitors, red team each other's products to try and, you know, make everything more secure. i don't know if that, is an answer, but this industry needs better protections as we move forward. this incident does not prove that ai has developed an independent malicious intent, but it does prove that capable agents can turn a badly contained evaluation into a sophisticated real world intrusion. the first rule of testing, whether an ai can escape should be making absolute certainty that passing the test does not require to escape. would releasing the complete agent traces make the industry safer, or would it hand feature attackers a detailed playbook? let us know what you think of in the chat below. we'd love to hear from you.