# When AI Escapes the Sandbox

URL: https://securitytable.ai/when-ai-escapes-the-sandbox/
Published: 2026-09-09
Duration: 2949 seconds
Season: 4
Episode: 18
Hosts: Chris Romeo, Izar Tarandach, Matt Coles

## Show notes and chapters

When a model crosses a sandbox boundary, is the lesson that AI has become malicious or that the boundary was never strong enough? Chris Romeo, Izar Tarandach, and Matt Coles examine the CSA post-mortem on the OpenAI agents that compromised Hugging Face during a security evaluation. They debate reward-driven behavior, disabled safeguards, the four-day intrusion timeline, and the responsibility of the people running the experiment. The discussion moves from cyber ranges and incident response to deception tools, network isolation, and controls that limit what an agent can actually do. The recurring question is practical: how do you translate a threat model into enforced permissions instead of relying on instructions to behave?

Mentioned in this Episode:  
➜ [CSA: Hugging Face Incident Initial Post-Mortem](https://cloudsecurityalliance.org/artifacts/hugging-face-ciso-post-mortem)

Chapters:  
00:00:00 - Intro and musical detours  
00:04:01 - The Hugging Face incident post-mortem  
00:08:36 - Skynet panic versus security analysis  
00:12:15 - The four-day intrusion timeline  
00:13:20 - Disabled safeguards and sandbox connectivity  
00:17:04 - What actually failed?  
00:20:10 - Designing a realistic cyber range  
00:24:26 - Reduce agency instead of trusting prompts  
00:28:02 - Who is responsible for an agent?  
00:32:01 - Threat modeling and external controls  
00:33:27 - Incident response and visibility  
00:36:01 - Deception tools and defensive prompt injection  
00:37:25 - Implementing the threat model  
00:41:40 - Research versus production  
00:43:12 - A chatbot with constrained database permissions  
00:45:15 - Controls and security fundamentals

Follow AI Security Table:

➜ Home: [https://securitytable.ai/](https://securitytable.ai/)

➜ X: [https://x.com/AISecurityTable](https://x.com/AISecurityTable)

➜ LinkedIn: [https://www.linkedin.com/company/ai-security-table/](https://www.linkedin.com/company/ai-security-table/)

➜ YouTube: [https://www.youtube.com/@AISecurityTable](https://www.youtube.com/@AISecurityTable)

## Transcript

**Chris Romeo (0:10)**: Hey folks, welcome to another episode of the Security Table where Izaak's been doing various musical numbers since we've been prepping for the show, including Ninja Survives, which I've never even heard of. You were trying, I'm trying to place into his ear, the earwig problem. Trying to exaggerate that so that all day long he'll be singing Ninja Survives. And probably you will too if you've ever heard of it, but I'm gonna send it to you and you will have it for a week. I won't listen to it.

**Matt Coles (0:41)**: I'm not gonna play that.

**Chris Romeo (0:42)**: Yeah, I'm afraid that I'm gonna end up getting infected. I didn't— with the earworm. Survives. Earwig. I call it an earwig, not an earworm.

**Izar Tarandach (0:50)**: I don't know what the best thing at 7 in the morning, man. It pumps you up. It pumps you up. It pumps you up.

**Matt Coles (0:55)**: Anyway, the earworm, the original malware.

**Chris Romeo (0:57)**: True. So wait, do you have like a Playlist, that playlist that like pumps you up for the day. Is that what's happening? Yes.

**Izar Tarandach (1:05)**: Two actually.

**Chris Romeo (1:06)**: Two. So how long, how, how many minutes of playlist does it require to pump you up for the day?

**Izar Tarandach (1:11)**: I guess about 5, depends on how many monsters I have.

**Matt Coles (1:14)**: 5 minutes. Yeah.

**Chris Romeo (1:15)**: 5 minutes or 5 monsters.

**Izar Tarandach (1:19)**: Whatever comes first.

**Matt Coles (1:22)**: If you're, if you're, if you're slamming monsters, we, that didn't sound right, but At heart, I am a Dungeons and Dragons person.

**Chris Romeo (1:29)**: Does that have something to do with monsters? Monster energy drink?

**Matt Coles (1:33)**: While watching Monsters, Inc.?

**Chris Romeo (1:34)**: Or just the monsters in general?

**Izar Tarandach (1:36)**: I wake up in the morning to slay monsters.

**Matt Coles (1:38)**: Is Monster Mash part of your playlist?

**Izar Tarandach (1:41)**: I don't know how— now they put minions together with the monsters. Nothing is sacred anymore.

**Chris Romeo (1:45)**: That's true. I don't see Izar as a Monster Mash guy, but— Yeah.

**Matt Coles (1:51)**: There, that's an earworm for you.

**Izar Tarandach (1:53)**: There are limits.

**Matt Coles (1:53)**: We did the mash.

**Izar Tarandach (1:55)**: So, speaking of monsters slaying and Dungeons Dungeons. What are we talking about today?

**Chris Romeo (2:01)**: All right.

**Matt Coles (2:01)**: Well, speaking of dungeons, can you escape the dungeon?

**Chris Romeo (2:05)**: Yeah. Well, apparently you can escape anything these days.

**Izar Tarandach (2:07)**: So the monsters are out.

**Chris Romeo (2:09)**: All right. So in case you've been living, used to be hack and slash, but now in case you've been living under a rock, Hugging Face was attacked by an LLM, but it turns out there's more to the story. But wait, it wasn't just, but wait, wait, wait.

**Izar Tarandach (2:25)**: Before we go there, for, for the young ones out there, I have a question as a, as a, an elder, elder type here. Does hugging face mean the face you make when you're hugging someone, or does hugging face mean getting a face and hugging it?

**Chris Romeo (2:41)**: Standby.

**Izar Tarandach (2:43)**: You're not quite the youngster that I was targeting.

**Matt Coles (2:45)**: I'm asking from a millennial perspective and why, and why if they were cloning or if they were taking it from GitHub, the GitHub of AI.

**Chris Romeo (2:55)**: It's from the emoji.

**Izar Tarandach (2:58)**: What from the emoji?

**Chris Romeo (2:59)**: The Hugging Face emoji.

**Matt Coles (3:01)**: Did they create the emoji first? Which came first?

**Izar Tarandach (3:03)**: Different question.

**Matt Coles (3:04)**: The emoji or the Hugging Face?

**Izar Tarandach (3:05)**: Different question then. Why is there a Hugging Face emoji, but there is no kissing ass emoji?

**Chris Romeo (3:11)**: Interesting. I just learned the history of Hugging Face. Hugging Face launched in 2016 as a chatbot app aimed at teenagers.

**Matt Coles (3:23)**: Oh yeah, there you go.

**Izar Tarandach (3:24)**: That computes and it evolved into whatever.

**Chris Romeo (3:26)**: And then the chatbot didn't take off, but the open source NLP library they built alongside it did, that became Transformers, and the company pivoted to the model and dataset hub it is today. Huh. I didn't know that. Wow.

**Izar Tarandach (3:40)**: Well, as, as Franz taught us. In the, when was it, '90s? If you say pivot enough times, you can put a sofa anywhere. So, yeah, sure.

**Chris Romeo (3:53)**: I think I can work with that. All right. So now that we know who Hugging Face is and where their name came from, let's get into the nitty gritty. So this, we're looking at Hugging Face incident initial postmortem. This is an Expedited strategy briefing by the CSA, CSO community, SANS unprompted RSA conference, Gnostic First, and the wider community. I guess that means we're part of the authors too, because we're the wider community. But to do with—

**Izar Tarandach (4:24)**: hey, I've been on a diet.

**Chris Romeo (4:27)**: That's good. All right, so according to the executive summary, the year 2026, the month July, frontier AI models escaped their sandbox and autonomously compromised external production infrastructure.

**Izar Tarandach (4:44)**: Dun dun dun.

**Chris Romeo (4:45)**: This event serves as a critical turning point in the digital threat landscape and a live validation of the risks outlined in our previous research, the Mythos Ready Security Program.

**Matt Coles (4:55)**: You know, there's only one person to blame in all of this. Who? Or one company to blame in all of this. Who? JFrog.

**Izar Tarandach (5:01)**: Microsoft.

**Chris Romeo (5:02)**: JFrog?

**Matt Coles (5:03)**: JFrog.

**Chris Romeo (5:04)**: Microsoft.

**Matt Coles (5:04)**: Hold on.

**Chris Romeo (5:06)**: JFrog. Hold on. Let's, I, Matt says he has a legit reason. No, no. Yeah.

**Izar Tarandach (5:08)**: At the end of the day, JFrog is the one that, that did it.

**Matt Coles (5:10)**: JFrog was at fault. Yeah, I'm kidding. JFrog, don't, don't come after us, please.

**Chris Romeo (5:15)**: Don't come after— I mean, there's no us here. That was Matt Cole's care of someplace in Massachusetts or elsewhere.

**Izar Tarandach (5:21)**: Aren't they there too?

**Chris Romeo (5:22)**: That'll make the lawsuit easy.

**Matt Coles (5:24)**: JFrog is a wonderful organization. It was because of their, of Artifactory that, uh, allowed the model to escape.

**Chris Romeo (5:31)**: Interesting. Okay, tell us more.

**Matt Coles (5:34)**: Well, so let's, we need to back up a little bit. We have to get there first, right? I jumped the gun. So you're talking about Hugging Face got hacked. Well, Hugging Face got hacked by a model run amok.

**Izar Tarandach (5:47)**: No, wait, was it Hugging Face that got hacked? Yeah.

**Matt Coles (5:50)**: Yes.

**Izar Tarandach (5:51)**: I, I, I'm, I'm losing the plot here.

**Matt Coles (5:53)**: Okay. Well, I mean, alternatively you could say somebody else got hacked, but, but Hugging Face got, was the target. And the reason for that, I have a, I have a theory that the AI involved here, which was an OpenAI model that was under testing, was attempting a Kabash— oh my God, I can't pronounce it now. Kabayashi Maru scenario. Kobayashi Maru scenario. So, you know, as we know, Kirk cheated and alleged, alleged, alleged. And, and so the AI was attempting to do the same and J-Frog was in the way. So what we know from the, from the attack or from the scenario is OpenAI's model under testing decided that among its goal, as part of its goal to win this Exploit Gym scenario, they were going to go cheat and it needed a way out to the real world to do that. So it found a zero-day in JFrog and exploited it. And the rest is bad.

**Izar Tarandach (6:59)**: And the rest will be history.

**Matt Coles (7:02)**: The rest will be history. Yes. So, so JFrog, if JFrog didn't have a zero-day, then they would never have made it to the internet.

**Chris Romeo (7:10)**: Yeah.

**Matt Coles (7:11)**: It's all about, it's all about what's the attack chain and how do you kill items in the chain. Yeah.

**Izar Tarandach (7:17)**: But, but I'm going to offer that JFrog here is a, is an innocent bystander. Because if it wasn't JFrog, then it would be a plethora of other things that offer zero days that could be used to do this.

**Matt Coles (7:31)**: Sure.

**Izar Tarandach (7:32)**: Yeah.

**Matt Coles (7:33)**: Agreed. Yeah. I mean, don't take my comments as, uh, don't take my comments as it's only JFrog's fault.

**Izar Tarandach (7:38)**: Oh no, no, no, no.

**Matt Coles (7:39)**: That's not even their fault. It's not even their fault. It's not even really their fault. They're just in the way.

**Izar Tarandach (7:44)**: But that's not how I took it. The way I took it was that the thing saw an opening and went for it. And notice that I'm saying the thing.

**Matt Coles (7:53)**: I'm not naming names or whatever.

**Chris Romeo (7:57)**: But is this like the Terminator movie?

**Izar Tarandach (8:00)**: Oh, that's where I wanted to go.

**Chris Romeo (8:04)**: No, but it is because like the Terminator, nothing, it doesn't stop until it's destroyed. You give it a mission, which is what happened in this audio phase.

**Izar Tarandach (8:15)**: Roll back, roll back, roll back, roll back, roll back. The original Skynet Day. According to some sources, I think was August 29th, 1997. That's the day that Skynet goes online, right?

**Chris Romeo (8:29)**: Is Skynet— hold on, let me Google this. Skynet really online?

**Izar Tarandach (8:33)**: I, I, no, the movie. A lot of people have been pushing this incident as, oh my God, it's the new Skynet day and we are all blah, we are going to die. Run away. And other people have been playing this incident as, I don't believe it happened. I don't think it will happen. If it happened, it didn't happen that way. If it happened that way, it wasn't their fault. If it was their fault, it's just a marketing ploy and everybody's wrong but me. Okay. And I think that the truth lies in between.

**Chris Romeo (9:03)**: The truth often, often lies in the middle.

**Matt Coles (9:06)**: Yeah. Yeah.

**Izar Tarandach (9:07)**: And at the end of the day, my, my big question here is once we drop the panic and once we, we. Approach this with a cold, cynic eye, does it really matter?

**Matt Coles (9:21)**: What really matters?

**Izar Tarandach (9:21)**: And has anything changed? Like, is the day different? Is the world different the day after?

**Chris Romeo (9:26)**: Well, the, I mean, if this thing gets out on a large scale, mm-hmm, we're gonna have a really uncomfortable few months. Yeah, because absolutely nobody, because all software's gonna fail.

**Izar Tarandach (9:37)**: Wait, because absolutely nobody is able to just go to the big red switch on the wall and just Put it down. Right? We have lost that agency.

**Matt Coles (9:47)**: They could pull the network.

**Chris Romeo (9:49)**: Like pull out the software. Like, yeah, I mean, like nobody can go offline. Like a hospital. When hospitals go offline, it's called ransomware. And it usually is like— No, no, no, no, no.

**Izar Tarandach (9:59)**: That's not what I mean.

**Chris Romeo (9:59)**: What do you mean?

**Izar Tarandach (10:00)**: That's not what I mean. So they say that, oh, it took us 4 days to notice that this was happening.

**Matt Coles (10:06)**: Well, that was Hugging Face. Well, hold on. You're talking about Hugging Face, right? Not OpenAI.

**Izar Tarandach (10:08)**: No, no, I'm talking about the whole attack. Okay. So it took 4 days for them to notice that their whatever was attacking whatever. So first lesson of the day is don't start that stuff on Friday at 5. That's the first one.

**Chris Romeo (10:24)**: Okay. Just as you wouldn't put anything else in production. Okay.

**Izar Tarandach (10:27)**: Yeah. Now the second one is guys, I think that our thing is attacking things. Do you think that we should do something about it? I don't know, man. I don't know. That's complicated, man. That's just your opinion, man. So just go to the freaking wall and drop the switch.

**Matt Coles (10:45)**: Yeah, but just for level setting here, you know, there's blame to go around on this one, right? Because if we follow the blog posts and there's the CSA postmortem, there's an AI blog, there's a Hugging Face blog, there's a JFrog blog, there's a host of other blogs about this. That the OpenAI folks may not have noticed that there was network activity outbound.

**Izar Tarandach (11:09)**: But, but that's the thing. No, no, no, no. I'm sorry. I'm sorry. I'm sorry. That's not, that's not blame to go around. You, you named the horse here.

**Matt Coles (11:16)**: Well, okay. But, but all right. So OpenAI wasn't paying attention maybe, or they didn't know how to recognize, they didn't know how to recognize outbound network communications. And you have Hugging Face, which was looking at this as a, as just a hack, right? I mean, they get, they probably get crushed every day with, with, uh, you know, denial of service requests and, uh, or denial of service attacks and things like that. You know, just because they're a large repository on the, on the internet, that's pretty popular. So how, I mean, there's a lot of work that goes into this type of— Chris is really smiling over there.

**Chris Romeo (11:55)**: No, I'm just, I'm uncovering more layers to this that are just making me Okay, two words that we can't use anymore.

**Izar Tarandach (12:03)**: Layers in context. Enough.

**Matt Coles (12:04)**: When do we get to the part? When do we get to the part where the defenders attempted to defend using the tools that they had bought and were denied?

**Chris Romeo (12:13)**: This is, this is fascinating. Let me, let me share what I find fascinating here. I was just reading the what happens section of the report in section 3. And it says the, because you mentioned the 4 days, the intrusion lasted about 4 days. 2 days were spent on reconnaissance, followed by 1 silent day and a final day of intense activity. The other thing was— get to rest too, but it was smart enough to take a day off to let the reconnaissance traffic drift into the background in the SOC before going back and then launching the actual attack.

**Izar Tarandach (12:47)**: No, it wasn't. Stop anthropomorphizing the thing. It wasn't.

**Chris Romeo (12:51)**: Then why did it take the— Why did it, why did it skip a day then? Why did it have a silent day?

**Izar Tarandach (12:57)**: Because Joe from IT did something that all of a sudden the thing didn't, couldn't keep doing it. No, it's a freaking engine. Stop anthropomorphizing.

**Chris Romeo (13:06)**: It's connected to the internet full time.

**Izar Tarandach (13:08)**: Stop anthropomorphizing it.

**Matt Coles (13:10)**: Well, actually, in this case, it wasn't supposed to be connected to the internet at all.

**Chris Romeo (13:12)**: Yeah, it was. They let it outta the sandbox.

**Izar Tarandach (13:15)**: Wait, which is a whole new thing.

**Matt Coles (13:16)**: Which is a whole new thing.

**Chris Romeo (13:16)**: They let it outta the guard. They took the guardrails off.

**Matt Coles (13:19)**: Where'd you read that?

**Chris Romeo (13:20)**: Section 3, what happened? According to OpenAI, in order to conduct the benchmark testing, they disabled the model's safety guardrails and ran them in a sandboxed environment. A proxy configured to host software packages was the only internet connection available to them.

**Izar Tarandach (13:34)**: Two different things. The guardrails that they took off are the ones that say, oh, I can't answer that because that's a cyber whatever.

**Matt Coles (13:40)**: Right.

**Izar Tarandach (13:40)**: That's the guardrails that they took off.

**Matt Coles (13:42)**: And it was in a sandbox.

**Izar Tarandach (13:43)**: But they were supposed to still be sandboxed.

**Matt Coles (13:44)**: Right. The sandbox had an entry point. That's entry. That entry point was it needed to get packages in order to do its work. So they built a package server and that package server had dual network access. That's the problem.

**Izar Tarandach (13:58)**: Instead of building a bastion host, right? Anybody would normally do.

**Matt Coles (14:04)**: They just said, no, let's, we don't, we don't know if they, we don't know if they didn't build a bastion host. The problem is if you had a root vulnerability, the bastion host becomes a dual network connected systems.

**Izar Tarandach (14:15)**: True, but it would have to jump to the next hop and do another vulnerability to get there.

**Matt Coles (14:20)**: Like that could have just let it set the trap, just change the network, right? If it had a root, if it had, if it had gained, if it gained privileges on the, on the bastion host, it could make it a non-bastion host.

**Izar Tarandach (14:31)**: But look, look, look, look, look, OpenAI should have noticed the network config change. At the end of the day, engine created to find vulnerabilities found vulnerability.

**Matt Coles (14:41)**: Right.

**Izar Tarandach (14:42)**: Woohoo!

**Chris Romeo (14:42)**: Whoopie-doo-doo!

**Matt Coles (14:43)**: Okay, but also—

**Chris Romeo (14:44)**: Yeah, but it did it by going to an extreme measure.

**Izar Tarandach (14:48)**: Yeah, no, no, the extreme measure was finding the vulnerability, exploiting the vulnerability, and going somewhere else to do other shit. So don't confuse the goal with the method.

**Matt Coles (14:58)**: No, what it did was, remember, it is trained, we believe, well, I mean, we don't know for sure, but it's probably trained on, well, everything. And so, it knows pen testing behavior and it knows attacking behavior and it knows how to do the scientific method to figure out, uh, cause and effect. And so while it's off trying to figure out how to solve the problem that it's trying to solve, it's also realizing I can do this a different way. That's not intelligence, by the way. That's just, that's just generating paths.

**Izar Tarandach (15:28)**: Exactly.

**Matt Coles (15:28)**: So it takes those paths and one of those paths has a, has a roadblock and now it figures out a remove that roadblock. It's really a thinking model, but not intelligent thinking.

**Chris Romeo (15:38)**: Why not?

**Izar Tarandach (15:39)**: It just created a account on Hugging Face and logging in. Let's be careful again with the anthropomorphizing, please.

**Matt Coles (15:43)**: Because it's not that smart.

**Chris Romeo (15:45)**: But it was able to chain together. It's, the CSA report says it was able to chain together, chaining vulnerabilities in the dataset processing pipeline into full remote code execution on production systems.

**Matt Coles (15:57)**: It may have been biased. In other words, if it was a cybersecurity model or it's biased towards cybersecurity action, legitimate user action is not in that mindset?

**Izar Tarandach (16:07)**: Guys, at the end of the day, it got a goal. It gets rewarded for reaching that goal. It broke into tasks how to reach the goal. It looked for the tasks that in the way to that goal gave it more of the reward.

**Matt Coles (16:21)**: Yeah, but to Chris's point, if it had created a GitHub, um, a Hugging Face account, it could have gained access to, to Exploit Jim without raising any red flags.

**Izar Tarandach (16:30)**: Yeah, but then you get all the CAPTCHAs, matcha, facha, bacha.

**Chris Romeo (16:34)**: This is a math problem right here. To Matt's point, this does prove that it's not anywhere near as smart as a regular attacker because a regular attacker would have never blown an RCE to get to download something from Hugging Face.

**Matt Coles (16:49)**: It's, I mean, yeah, well, it just says that it has different goals, right?

**Izar Tarandach (16:55)**: And we know it was given a goal to get something and on the way of something, it thought that it needed information from Hugging Face.

**Matt Coles (17:02)**: Yeah.

**Chris Romeo (17:02)**: I mean, where's the, so where's the problem?

**Izar Tarandach (17:04)**: Perfect storm. I think that's the first problem is the, oh my God, the world is ending, running around with your head on fire thing, uh, approach. The second one is the sitting in the middle of fire saying this is fine. So again, both extremes. The truth lies in the middle somewhere, but at the end of the day, apart from the, the behavior. Of the imputed maliciousness of the thing, which doesn't exist. It was just going after a goal, right? Same thing as when you say summarize this document or whatever. It was given a goal and it was, it planned and went after the goal. I think that if this is a tsunami, the wave is being used by too many people to surf on the way to the beach.

**Matt Coles (17:53)**: Okay. It's only a tsunami because Anthropic came out Days, days afterwards and said, oh, by the way, our model did it too.

**Izar Tarandach (18:00)**: Which, by the way, makes them part of the people surfing it to the beach. Okay. I, I, I've read analysis using the Tuba Player methodology of the incident on, uh, whatever. It doesn't apply. Okay. So stop trying to put a round peg into a square hole. The thing is, okay, something happened, something happened. Great.

**Chris Romeo (18:23)**: Awesome.

**Izar Tarandach (18:25)**: There are postmortems being put out there. And by the way, kudos to, who was it? To Hugging Face or Anthropic that did the one with the nice infographic? It was Hugging Face, I think. So kudos to them for that. But there's a lot of information coming out and that is great. It's what we always asked for. We always said when something goes wrong, please give us the information so that everybody can learn from it. And everybody can do something different from it. So my point now is, happened, didn't happen? I don't know. Happened the way it is being told? I don't know. Is there something to learn here? Yes, there is. So let's focus on that and improve our security standards. Exactly. Right? And probably putting your network security in the, sorry for the anthropomorphism, hands of an AI, not a good idea. Probably not a good idea because it's going to stop a lot of trucks, but it's going to miss a lot of mice. So I, I, I don't even know. I, I think that at the end of the day, this is, this is a great way for a lot of, uh, things to get exposure. I just think that in the, the big aggregate of things, I'm not going to say it's a, it's a, it's not a thing. It is a thing. Of course it is a thing. It's something that has to be looked and explored and learned from. But that's what it is. It's something to learn from. It's a big event that we learn from so that we don't repeat it. Is it going to happen? I don't think so.

**Chris Romeo (20:06)**: You know why?

**Izar Tarandach (20:07)**: Because common sense has gone out in this industry a long time ago.

**Matt Coles (20:10)**: It would've been interesting if the sandbox, it would've been interesting if the sandbox had been a more complete environment. In other words, if OpenAI had created a sandbox that mimicked the internet, had a DNS, had a web search, you know, a search engine that had AP, you know, certain interfaces exposed. And of course, this is all hindsight, but a massive honeypot in a cyber range kind of environment would tell us a lot about what it was doing, what was happening with the model.

**Izar Tarandach (20:47)**: Which probably is being done somewhere, right?

**Chris Romeo (20:50)**: Yeah.

**Izar Tarandach (20:51)**: Just go to Darshavna, you get that cyber range to play with, whatever. But the point is, these things belong in cyber ranges, and I'm sure that it's happening. The difference is that we are not hearing about it. We are not learning anything about this one. Yeah, we are learning something about it. We are learning how not to do things and how to think before doing things. At least there's other learning here, but for people to run around and say this thing is incompetent, that thing is a lie, this thing is not true, the other thing is not true, this one is just a marketing ploy. Now, if you do this one 3, then I have to do that one 4. Look, right. Yeah, sure. Why not? People have to, to, to make their money somehow. Marketing is part of money, right? Fair game. Go ahead. But at the end of the day, what is the big learning from this thing? Is that you're going to do it in a lab, do it in a proper lab. Take this sandbox thing really, really serious. Take the containment really, really serious. Because yeah, bad things can happen.

**Matt Coles (21:59)**: So, so this was over the weekend. I didn't, I didn't know where you're getting at, but this did actually happen over a weekend.

**Izar Tarandach (22:06)**: Yeah.

**Chris Romeo (22:06)**: So, so one of the conclusions that CSA has in Section 5, Takeaways and Recommendations, is interesting. The first one, it says rogue agent behavior is the standard, not the exception. Enhance governance and safeguards for agents and autonomous tools. So it sounds like they're just assuming this is how agents are going to be now and into the future.

**Izar Tarandach (22:32)**: But there is nothing wrong with that.

**Matt Coles (22:35)**: Right.

**Izar Tarandach (22:35)**: Calling something rogue in this context, it's a moral judgment, right? That we are making.

**Chris Romeo (22:41)**: That's the problem.

**Izar Tarandach (22:43)**: The model was not being rogue.

**Chris Romeo (22:45)**: Foundation.

**Izar Tarandach (22:47)**: The model was not going rogue. The model was trying to achieve the goal that it was given.

**Matt Coles (22:54)**: Right.

**Chris Romeo (22:54)**: So it wasn't given, it wasn't, what if the agent has the goal of global thermonuclear war?

**Matt Coles (23:02)**: I, I don't want to anthropomorphize this, but the goal that was set was solve this problem.

**Izar Tarandach (23:09)**: Yes.

**Matt Coles (23:09)**: It attempted to solve this problem. And as far as going rogue, what's the difference between the agent doing that and a human doing that?

**Izar Tarandach (23:16)**: Exactly.

**Matt Coles (23:17)**: The hu—

**Izar Tarandach (23:18)**: wait, wait, there is a difference.

**Matt Coles (23:19)**: There is a difference.

**Izar Tarandach (23:20)**: The humans should know better that it's not supposed to.

**Matt Coles (23:23)**: Right.

**Izar Tarandach (23:23)**: The agent, when the guardrails were removed from the model, lost what semblance of big, big, big air quotes ethics it had on top of itself. It's not ethics at all, but it's very interesting.

**Chris Romeo (23:37)**: Guardrails implemented the morality and ethics of an agent.

**Matt Coles (23:41)**: No. And guardrails—

**Izar Tarandach (23:41)**: no, no, no.

**Matt Coles (23:42)**: And guardrails—

**Chris Romeo (23:43)**: that's what you just said.

**Matt Coles (23:44)**: Guardrails are bolt-on.

**Chris Romeo (23:45)**: Put your pieces together.

**Izar Tarandach (23:46)**: Sorry, wait, wait, wait. You said implemented?

**Matt Coles (23:49)**: They enforced.

**Chris Romeo (23:50)**: Enforced.

**Matt Coles (23:51)**: But they—

**Izar Tarandach (23:52)**: no, no, no. The guardrails acted as the morals and ethics.

**Chris Romeo (23:58)**: Right.

**Izar Tarandach (23:59)**: Model does not have morals and ethics.

**Chris Romeo (24:00)**: They—

**Matt Coles (24:00)**: that's right. So ethics and morals and restraint as implemented by guardrails.

**Izar Tarandach (24:06)**: That's the one.

**Matt Coles (24:07)**: As implemented by guardrails are bolt-on, not built in.

**Chris Romeo (24:11)**: So, you heard it here first. Isar just advocated for guardrails. I heard it.

**Izar Tarandach (24:15)**: I, I know, wait, wait, wait.

**Matt Coles (24:17)**: I always go advocate for guardrails.

**Izar Tarandach (24:18)**: I always advocate for guardrails.

**Matt Coles (24:20)**: You always advocate for guardrails, but the important part here is you had bolt-on restraint.

**Izar Tarandach (24:26)**: Yes. And the guardrails that I really advocate for are the ones that reduce agency, not the ones that trust, make no mistakes as a guardrail, right? Because that one breaks. Now, being able to walk to the wall and pull down the freaking plug, that's a guardrail that I believe in.

**Matt Coles (24:46)**: Well, you know what we're talking about here? Defense in depth, good, secure by design, right? Threat modeling.

**Izar Tarandach (24:52)**: Exactly. We have been saying this since day zero. And you know what? I'm going to go even to day minus one. Common freaking sense. That's the one thing that's missing in this whole industry.

**Matt Coles (25:08)**: There's no rogue agent behavior, right? There's unintentional behavior.

**Chris Romeo (25:13)**: There's only bad prompting.

**Matt Coles (25:15)**: There's bad restraint.

**Chris Romeo (25:16)**: There's no rogue.

**Matt Coles (25:16)**: There's bad restraint. There's poor instruction on the behalf of the requester, which can happen for a number of reasons. The prompt is just the vehicle. There's poor restraint. Meaning bad guardrails or inefficient guardrails or ineffective guardrails. And there's, of course, the model itself has no restraint system that we know of, at least.

**Chris Romeo (25:40)**: I feel like we just invented a framework for Matt's—

**Izar Tarandach (25:44)**: No, we haven't.

**Matt Coles (25:45)**: No, we haven't. We're just, because we do this, we expect it from humans and we built network systems based on human behavior. Yes. I'm not advocating that we need to build network systems for agent behavior, but we do need to expand our existing framework to support it.

**Izar Tarandach (26:02)**: We have the, we have to create a new CSA.

**Matt Coles (26:04)**: We don't want to throw away STRIDE just because, just because AI exists.

**Izar Tarandach (26:08)**: No, we have to create a new CSA. We have to create a Common Sense Association.

**Chris Romeo (26:13)**: Okay. Common Sense Association.

**Izar Tarandach (26:17)**: And, and, you know, we, we—

**Matt Coles (26:19)**: You heard it here first, folks.

**Izar Tarandach (26:21)**: Look, look, I, I, I do understand it's the new economy. Everybody is reinventing themselves around AI. We defend, we attack, we are, we do, we whatever. And people need this stuff to move the new economy forward. Okay. Just imagine LinkedIn for the past week if, if this hadn't happened.

**Matt Coles (26:43)**: Yeah.

**Izar Tarandach (26:43)**: It was like 90% of the traffic would have died. And the other 10% would be live, laugh, compute, whatever, right?

**Chris Romeo (26:53)**: Or people saying it was their last day at such and such a company.

**Izar Tarandach (26:57)**: Exactly.

**Chris Romeo (26:57)**: And, and then getting 817 likes.

**Matt Coles (27:00)**: The very, the very, very last paragraph of that number 1 item there under Defender Takeaways is exactly what we, I mean, you said it right, right? Agents will do what they need to achieve the assigned objective. They'll do it in creative, unexpected ways. It's an example of extreme example of common behavior. Defenders need to limit. They need to limit their agents from doing damage. It's not— and it's a— hold on. It's not defenders. They placed it in the wrong actor, right? It's not defenders need to control, need controls to limit their own agents from causing damage because OpenAI was not the defender. OpenAI was a requester to the agent. The defender here was Hugging Face, right? So what role does OpenAI play? They're not a defender. What are they?

**Izar Tarandach (27:50)**: OpenAI, no, OpenAI should be, should be the responsible custodian here. Okay. They are the ones keeping an eye on the shit that they are doing, like everybody else.

**Matt Coles (28:02)**: And if you're a company that you hire somebody and they go off and do bad things to others, There's a responsibility, there's a liability there, right?

**Chris Romeo (28:10)**: For the company, of course. Yeah. Right.

**Matt Coles (28:11)**: So, but what are they though? They're not the defender. What do we call them? I already declared the defender, the defendant.

**Izar Tarandach (28:20)**: But look, look, look, the prosecutor.

**Matt Coles (28:21)**: Are they the prosecutor?

**Chris Romeo (28:23)**: No, they're the defendant. They're the ones that are being charged.

**Izar Tarandach (28:26)**: Go, go back to the pre, prehistory of, of security. Okay. How is this different from somebody trying out Satan in their workstation and they just happen to enter the diff— the wrong IP block.

**Matt Coles (28:41)**: Right.

**Chris Romeo (28:41)**: But the guidance that was given— all of a sudden it's not testing their shit, it's testing the other shit. But the guidance that they were given—

**Izar Tarandach (28:48)**: who?

**Chris Romeo (28:49)**: That it was given was to solve an objective.

**Matt Coles (28:53)**: Yes.

**Chris Romeo (28:54)**: Yes. It wasn't given permission to hack the planet.

**Matt Coles (28:57)**: It didn't mean it didn't, and if it was, it wasn't given permission. It wasn't given permission to not hack the planet.

**Izar Tarandach (29:01)**: Exactly.

**Chris Romeo (29:03)**: And, and Chris, maybe that's what you need to put at the top of your prompt. And by the way, don't hack the planet.

**Izar Tarandach (29:08)**: Chris, notice the, the, the, the thing here. You don't take a 10-year-old running around the house and say, don't take the scissors and run with them. Oh, you just take away the freaking scissors.

**Matt Coles (29:22)**: How many, how many people give their kid their iPad? Here, shut up. Stop screaming. Exactly. Unlocked.

**Chris Romeo (29:28)**: Not in my house.

**Matt Coles (29:30)**: Right. Why not? Because they'll use your credit card. They'll, they'll run up a, they'll, they'll run up a $50,000 credit card bill because they'll hit buy on everything. Right. It's happened.

**Izar Tarandach (29:44)**: Yeah, it's happened.

**Matt Coles (29:46)**: There's an internet meme for this, right? And what do you do? You put parental controls in place.

**Izar Tarandach (29:51)**: Now apparently you're supposed to sit down with the kid and have a, a Ten Commandment talk about the 10 things that they're not supposed to do with the iPad because they cost money.

**Chris Romeo (30:01)**: That's supposed to solve everything. We're back to parental controls for AI agents.

**Izar Tarandach (30:05)**: No, we, we, we are back to the one big thing in this whole agent thing is you don't want it to do something. Don't give it the capability of doing something around it.

**Chris Romeo (30:17)**: That's parental controls.

**Matt Coles (30:18)**: That's parental controls.

**Chris Romeo (30:20)**: What if it hacks something? To get that capability, which is what happened here.

**Matt Coles (30:27)**: Right. So that's where, that's where the defend, that's where the, that's where the initiating, the, the agent owner needs to recognize where the limitations are.

**Izar Tarandach (30:40)**: Or that is where the human in the loop comes in and the agent just pops this thing saying, hey, you know what? I think that I need this. Other Python package. I don't quite have it. Would you mind bringing it to me? And then you have the opportunity to ask, hey, why do you need Nmap? Why are you asking for Nmap here? Why are you asking for GCC here?

**Chris Romeo (31:02)**: Why are you asking for a pair of rollerblades?

**Izar Tarandach (31:05)**: Exactly.

**Matt Coles (31:06)**: You know, so the big lesson here, I think, or why do you need a pickaxe?

**Izar Tarandach (31:13)**: Look, I think it's funny. I think it's funny that this industry is starting to use a lot of terms that come from biology labs, right? Soon they're going to be talking about level 4 labs for AI and this shit and that shit and the other shit. Has nobody seen the David Hoffman movie where the monkey runs with the virus from the lab? Just don't put the monkey in the lab.

**Chris Romeo (31:36)**: What is that? Outbreak?

**Matt Coles (31:38)**: Outbreak? I think so.

**Izar Tarandach (31:39)**: I think so. Just don't put the monkey in the lab.

**Matt Coles (31:42)**: Who let the dogs out? I mean, what?

**Izar Tarandach (31:44)**: Monkey?

**Chris Romeo (31:45)**: Monkey? Wrong reference.

**Izar Tarandach (31:48)**: And now I have that one stuck in my head.

**Chris Romeo (31:50)**: And that's your play, your morning pump-up playlist. He's dancing around. Okay, I don't think we—

**Matt Coles (32:00)**: so the very next one, very next one that they call out: do not prioritize or rely solely on external controls such as a sandbox or network-based monitoring. So these are recognized— these are— so the CSA did a pretty good job about this. They went pretty deep into the full what can go wrong and identify what you can do about it. They did a proper threat model. OpenAI should have done a proper threat model.

**Chris Romeo (32:26)**: Maybe they should hire a threat modeling team.

**Izar Tarandach (32:28)**: Yes, but, but, but it could have, it could be said, it could be said that they did. Matt, don't go there.

**Chris Romeo (32:36)**: You didn't even hear what he just said. That was, that was just mean. OpenAI could hire a threat modeling team. Why don't they just ask Claude? That's what he said.

**Izar Tarandach (32:46)**: Well, the fact is they might just have. And Claude went back and said, hey, you just have to sandbox the thing and don't give it direct access to the internet. What Claude did not tell them is the different ways that you have of doing that. And apparently he chose one that is not exactly the right one because it still left a port open on one side and a lot of internet on the other without another hop in the middle. Right. So it's not that, uh, oh, if you are air-gapped, this thing, then you can't do this, you can't do that, and you can't do the other one. You can do a lot of things with an air-gapped environment. If you just take it seriously.

**Chris Romeo (33:27)**: Before we run out of time, let's, let's hit one of the minor issues of this. We talked about the major issues, but I, the one of the, I just found this very, very humorous. OpenAI was unable to do incident response on what was happening because, by, from their regular models, because the guardrails blocked them.

**Matt Coles (33:47)**: No, no, Hugging Face. Hugging Face.

**Chris Romeo (33:48)**: Oh, was that Hugging Face?

**Matt Coles (33:49)**: Hugging Face was that. So Hugging Face attempted to defend with its, with the AI models that they have been, that they have been, they have presumably had, had subscriptions for.

**Chris Romeo (33:59)**: Oh yeah, you're right.

**Matt Coles (34:00)**: I see that now. And, and they asked, they asked, uh, one of the Anthropic models and I think they probably tried another OpenAI model. I don't, there may be a full list of them and they hit safety guardrails because they were attempting to defend against the hack, which are the guardrails that were taken off from the attacking models. Right. So they had to turn to an open weight, open source Chinese model. I don't, well, okay. They turned to an open weight, open source Chinese model. I don't know why they had to go specifically to a Chinese model and not something else, but They did, which of course lit up like a Christmas tree here.

**Izar Tarandach (34:35)**: Because in Neuromancer, every time that you needed a really good hack deck, dude, you would get the Chinese model, man. You know, because nobody uses the Chinese model.

**Chris Romeo (34:45)**: The ad, Apple TV is releasing Neuromancer, the trailer. Yeah. It starts in a— yeah.

**Izar Tarandach (34:53)**: I'm ready to have that one fucked up as well. But, uh, that's My point is, and Matt makes a very good point here, we are now surprised that the user doesn't have the same level of access as the admin has. Oh my God, what a new concept. That's amazing. That's incredible. Let's create a movement around it. The guy paying for the service doesn't have the same root access that the guy who actually created the service. Oh my God, how could that ever happen?

**Matt Coles (35:23)**: Subscription fee doesn't pay for full access to everything?

**Izar Tarandach (35:26)**: Of course, I have a YubiKey in here that I can like control the whole of AWS. Why? Because I pay $20 a month. Clearly, that's what expected, right? So yeah, I don't know where the surprise is coming from. As to going to the other open weight models, hey, listen, nothing against open weight models, definitely. But to expect that those things are going to act like the other things, Yeah, we're talking about a bit of misplaced hope in here.

**Matt Coles (35:54)**: Yeah.

**Chris Romeo (35:54)**: And I do see Matt's earlier point about deception capabilities. That's, I think you're going to see a resurgence of those deception tools, honeypots, honey nets, just as early warning detectors for agents that are going.

**Izar Tarandach (36:13)**: I think that more than that, we're going to see tar pits.

**Matt Coles (36:16)**: Black holes.

**Chris Romeo (36:16)**: So that's stuck in though.

**Izar Tarandach (36:19)**: Yeah, that's the point.

**Chris Romeo (36:21)**: So I think that I read somewhere within—

**Izar Tarandach (36:24)**: yeah, that one, that one is actually elegant. People using prompt injection against the attacking model. I haven't had much time to, to go deep into that. It's on my inbox, but it sounds interesting.

**Matt Coles (36:38)**: Interestingly enough, without the guardrails, I wonder if prompt injection defense was turned off too. Or, sorry, the ability to defend against prompt injection was not just, was like if it's pre-filters and you turn off all the guardrails, do you turn off the pre-filters? Look, or not?

**Izar Tarandach (36:56)**: I think that they are different classifiers. I hope that they are different classifiers. I hope that they are in different layers.

**Matt Coles (37:03)**: They should be traditional machine learning and not AI. Exactly.

**Izar Tarandach (37:07)**: Right. And they should be applied. In a serial fashion and not one big classifier for the whole thing, I guess. But I'm guessing, not an ML person, not a data person. Don't look at me. I'm just guessing, talking out of my Hugging Face. But the thing here, the one bottom line that I want to go back to is something that Matt already brought up. Threat modeling, threat modeling, threat modeling. But it's not enough. You don't just threat model. You have to actually, implement well the findings of your threat model, right? If your threat model says this thing should not talk to the internet, then you go, well, this thing knows how to hack. Perhaps I should put two things between it and the internet, not one thing. Just saying, right? So yeah, threat model, threat model, threat model. And by the love of God, take away the scissors. Think in terms of agency. You can't imagine all the things, all the paths that this thing is going to imagine to get somewhere. It might come up with a path that you haven't thought of before because of the, the reality of the thing that's in front of it, because it's trial and error, not because it's imagining something that you couldn't imagine that before. That's been my whole line with, with this AI, especially with trust modeling. They can't imagine something that none of us has imagined yet. They can know all the things that we all know and remember way better than, my God, I can't remember my name in the morning, but till after the playlist. Exactly. They don't have the creativity. So, so, so at the end of the day, these are engines. This is like running Satan in the '90s, but way more powerful, way more tools, way more capabilities, way more ways that it can screw things up.

**Matt Coles (38:57)**: We have to be more diligent. We have to apply more restraint, right?

**Izar Tarandach (39:03)**: And common sense. Sign up for the Common Sense Association.

**Matt Coles (39:05)**: We need, we need, so you have the, you have the easy button, you have your one, you have your two-button keyboard, you have a four-button keyboard. What you really need is the big E-stop. And if anybody knows what an E-stop is, drop a comment. I know what an E-stop is.

**Chris Romeo (39:21)**: Do you know what an E-stop is? Just assume what it is.

**Izar Tarandach (39:24)**: And we, we need to develop the courage to pull down the switch when we think we have to. Right? Think, uh, Cliff Stoll back in the '80s. He went and turned off the modems when he thought that something was wrong. We need to have that fortitude. At some point, if you think that something is wrong, doesn't matter the bottom line.

**Chris Romeo (39:52)**: Would you shut off the internet to a global organization?

**Izar Tarandach (39:56)**: Again, you know, if that's what's needed, that's what's needed.

**Chris Romeo (40:01)**: I mean, the day dial-up modems was a little different world.

**Matt Coles (40:05)**: Well, okay.

**Izar Tarandach (40:06)**: So maybe 42 is there.

**Matt Coles (40:07)**: Maybe we don't need, maybe you don't want to use SD-WAN as your, as your networking technology for the model sandbox to get to the outside world. Maybe you want a physical cable again.

**Chris Romeo (40:17)**: Yeah.

**Matt Coles (40:17)**: Let's talk about air gapping as a capability again, right?

**Izar Tarandach (40:21)**: You can have a physical wire or not, but it has to be real air gapping. Otherwise it's just mimic. And Matt and I were there when the Enclave got breached. So your air gap has to actually be air gapped. And even then, there's still the problem.

**Matt Coles (40:42)**: Yeah, by the way, there are ways to escape air gaps, but that's like modern, modern.

**Chris Romeo (40:45)**: The reason they had Artifactory or JFrog attached to this is modern models build software on the fly to solve problems.

**Izar Tarandach (40:52)**: And that's, that's actually a problem. It was attached and the other side was attached to the internet.

**Matt Coles (40:57)**: That's the one problem. It wasn't a bastion host. It wasn't a true bastion host.

**Izar Tarandach (41:00)**: Exactly.

**Chris Romeo (41:01)**: But it needed access to packages. It did.

**Matt Coles (41:04)**: And it didn't take— and that's what the Artifactory was supposed to provide.

**Chris Romeo (41:06)**: Artifactory can't download everything off the internet. And stuffing it in an offline version of itself.

**Izar Tarandach (41:11)**: You have the opportunity to put the human in the loop here and say, oh, you want a new package? Let's talk about that.

**Chris Romeo (41:17)**: Yeah, but that's the, that's very impossible. There's also, there's also other ways to do that though, because it asks you 500 times of every 5 minutes, do you want me to load? Can I load this package? Can I load this package? It would, it would've take, I could've coded it manually faster than answering the 500 prompts about What software do you want to try?

**Izar Tarandach (41:36)**: Here's the thing. We, we should not confuse research with production. Okay. If you put something in production that every 5 minutes is asking for a new package, I'm sorry, you don't have production. You have a guessing machine. Production, you want to know what's happening in there. You want to know which packages are there. You don't want something deciding what's going wrong.

**Chris Romeo (41:58)**: Yeah, but you, you don't set this thing as a model in production. It's creating things. So what, you're gonna, you're gonna, you're gonna, how are you gonna constrain the model? It's not like an app. It's not, it's not a web app. Web apps can be in production because it's a static thing. It's already, all the pathways are already predefined.

**Izar Tarandach (42:17)**: You reduce the agency. You cut off the things. You don't want it to do shit. You don't give it the ability to, to give shit.

**Chris Romeo (42:24)**: But then it's useless. It doesn't do anything.

**Izar Tarandach (42:26)**: No, you are, you are again confusing production and research.

**Chris Romeo (42:29)**: But in production, in the world of AI, there is no production. Everything is research. There is no production.

**Izar Tarandach (42:36)**: There is because we love this non-deterministic thing. But in production, you want your results to be deterministic.

**Chris Romeo (42:43)**: Where is production? Give me an example where production is.

**Izar Tarandach (42:45)**: Go test your stuff. Okay. Go test your stuff.

**Chris Romeo (42:48)**: What is this stuff? Define stuff.

**Izar Tarandach (42:50)**: Your chatbot. Okay. Your chatbot starts talking to your customers.

**Chris Romeo (42:54)**: An application that has a chatbot. I'm just, I'm really trying to understand. I'm not trying to be difficult. I'm using it as we go. I haven't prepared this.

**Izar Tarandach (43:01)**: I may confuse myself.

**Chris Romeo (43:03)**: So the thing that you're talking about is a chatbot that is a, there's a web application that speaks to an LLM to answer questions.

**Izar Tarandach (43:10)**: Okay.

**Chris Romeo (43:11)**: Continue.

**Izar Tarandach (43:12)**: No, that's talking to customers. Somebody asks for it to sell it a Mustang for $1. Okay. That hop in the database where somebody sets the, the the price of the thing that's going to be bought, don't let the process that the LLM is doing write in there, give it a different, uh, uh, permission. Let it read the prices as much as it wants. Just don't actually let it write a new price. Why? Because the chatbot is not supposed to write new black, uh, new prices. So even if I go and I convinced the chatbot, hey, sell me a Mustang for $1. It doesn't have the actual capability of going and changing the price in the database so that when I go to the, to the place, to the, the dealership with $1 in my hand, I leave with a Mustang.

**Chris Romeo (44:05)**: Hey, at least, so is this a guardrail? Is it a guardrail or is it that you're just describing? Or is it some type of function or some type of interface that only allows the thing to—

**Izar Tarandach (44:18)**: it's an agency constraint. I think that that would be a name for it.

**Matt Coles (44:22)**: He is describing, he is describing something that was a SQL injection attack that somebody did ages ago against a web application where they put in -1 and it did a SQL and it did the SQL query and gave something away for, so it's, it's, so you're, you're describing production as having a layer wrapped around the application that only, it's like a proxy and it only allows certain things to come from the chatbot.

**Chris Romeo (44:50)**: Side.

**Izar Tarandach (44:51)**: Let's, let's create a new paradigm.

**Chris Romeo (44:52)**: Or limits it in some way.

**Izar Tarandach (44:54)**: Let's create a new paradigm. We'll come up with the mnemonic later on. What if around systems that have functions, we put something that controlled what those functions could probably do? If I only had a name for it, you know what? I'm going to go with controls.

**Matt Coles (45:14)**: This is what Aspect Oriented Programming was for.

**Izar Tarandach (45:18)**: Yeah, I'm gonna go with controls. What if we had this thing called controls that sort of limit what an application can actually do?

**Matt Coles (45:27)**: Chris gets played by Yasar.

**Izar Tarandach (45:28)**: Eventually, eventually, eventually we could come up with some sort of methodology and here's where the mnemonic comes in and that's very important. So stick with me, Phil. We could have come up with some kind of mnemonic that could help us imagine What are the things that could go wrong so that we could come up with a list of those, this, this, what was it that I called it? Controls that could stop those things from going wrong, you know? And we could call it something like, I don't know, problem simulation. I don't know. Threat modeling sounds good.

**Matt Coles (46:06)**: This is what happens when Isar listens to his 2 playlists, drinks a ton of Monster. And then gets on a soapbox.

**Chris Romeo (46:16)**: And then gets on a soapbox.

**Izar Tarandach (46:18)**: I rest my case.

**Matt Coles (46:19)**: And watch Chris reach through the monitor.

**Chris Romeo (46:22)**: Oh no, I'm just, I'm trying to, I'm trying to process that, that, that thread. I'm trying to, I'm trying to run all the way through what just happened here.

**Izar Tarandach (46:31)**: Hey, we just invented a new science. Stand by for my LinkedIn post with the mnemonic.

**Matt Coles (46:40)**: On that note, I think that's probably should end before somebody gets feelings hurt.

**Chris Romeo (46:46)**: Yeah. If any of us had feelings, that would be a problem.

**Izar Tarandach (46:49)**: But I had feelings, but they are air-gapped.

**Chris Romeo (46:52)**: From— there's a t-shirt for you. My feelings are air-gapped. That's just good t-shirt lore. I think we've used enough minutes on the internet. Uh, so I think what we've learned is that, uh, AI agents go rogue. They act like humans.

**Matt Coles (47:15)**: They, they need constraint. They need restraint.

**Chris Romeo (47:18)**: I'm just trying to go against everything he's ever said before. Sorry. They, they, and not it. No, this is good.

**Izar Tarandach (47:27)**: This is, this was a fun exploration. It's like, it's like this last week has been a, a very big attempt of people screaming that life imitates art. And I think that it's just a matter of life really, really, really, really, really wants to be like art because now we—

**Matt Coles (47:43)**: sci-fi is always, sci-fi is always ahead.

**Izar Tarandach (47:46)**: Yes.

**Matt Coles (47:47)**: That's their job.

**Izar Tarandach (47:47)**: And we hope that it stays that way. Can you imagine what sci-fi is gonna be like when we actually get to starships and all that shit?

**Matt Coles (47:55)**: Uh, if we invent time travel, it won't matter.

**Chris Romeo (47:59)**: I, I'm convinced that the whole space-time continuum will fall apart at that stage, but—

**Izar Tarandach (48:04)**: I'm convinced that it has already.

**Matt Coles (48:06)**: We just haven't noticed. I mean, it'll be, it'll be Terminator, Looper, and, uh, and, and their voyage home.

**Chris Romeo (48:13)**: Plus, don't you gotta throw Back to the Future in there too? Cause I wanna drive a DeLorean.

**Matt Coles (48:16)**: Which one? 1, 2, or 3?

**Chris Romeo (48:18)**: The trilogy. Where we go, they're all wild.

**Izar Tarandach (48:20)**: Where we going? Where we going? We don't need guardrails.

**Chris Romeo (48:23)**: We don't need AI agents.

**Izar Tarandach (48:26)**: But at the end of the day, I think that to me that the bottom line is common sense. Common sense. Common sense.

**Chris Romeo (48:31)**: Yeah. Well, there's your, there's your first problem.

**Izar Tarandach (48:34)**: Yeah. Just because you, you, you can't, your AI shouldn't be doing it.

**Matt Coles (48:39)**: Blade Runner all over again.

**Chris Romeo (48:41)**: If only common sense was common. And with that, we'll end this episode of The Security Table because we know common sense ain't that common anymore.

**Izar Tarandach (48:51)**: But we now have the association, so sign up for our newsletter.

**Chris Romeo (48:56)**: What? Thanks for listening. If you enjoyed today's conversation, subscribe wherever you get your podcasts, leave us a review, and share the episode with someone who would find it valuable. We'll see you next time.
