# Stop Patching Bugs, Just Rewrite It… Says the Guy Selling the Tokens

URL: https://securitytable.ai/stop-patching-bugs-just-rewrite-it-says-the-guy-selling-the-tokens/
Published: 2026-11-11
Duration: 2983 seconds
Season: 4
Episode: 27
Hosts: Chris Romeo, Izar Tarandach, Matt Coles

## Show notes and chapters

Cisco's president handed a 27 year old engineer unlimited AI tokens and 24 months to rewrite 98 percent of the company's legacy code, and Sam Altman now says we can no longer just patch all the bugs. Chris sees a step toward disposable software, where you regenerate the thing instead of fixing it. Matt calls it the holy grail of automation and traces the dream back through UML, code generators and no code tools. Izar wants to know why anyone would bother, and points out that the loudest voice for rewriting everything is the guy selling the tokens. Then it is Hirundo's finding that Qwen3 Coder writes more hackable code when told the app is for a US defense contractor, ProjectDiscovery's backdoored abliterated model, and whether open weight models can be trusted at all. Izar's answer: start with a concrete use case, then verify, verify, verify. Plus: Izar admits he doesn't have enough of a life to need an AI agent.

Mentioned in this Episode:  
➜ [Cisco is letting a 27-year-old refactor 98% of its legacy code with unlimited AI tokens: 'Go nuts'](https://fortune.com/2026/10/30/cisco-president-jeetu-patel-gen-z-27-year-old-unlimited-ai-tokens-rewrite-legacy-code-entry-level-jobs/)  
➜ [Sam Altman: 'I No Longer Think We're Gonna Be Able to Just Patch All the Bugs'](https://gizmodo.com/sam-altman-i-no-longer-think-were-gonna-be-able-to-just-patch-all-the-bugs-2000829947)  
➜ [Bias in Qwen3-Coder Model](https://www.schneier.com/blog/archives/2026/10/bias-in-qwen3-coder.html)  
➜ [How abliterated models can get you pwned](https://projectdiscovery.io/research/how-abliterated-models-can-get-you-pwned)

Chapters:  
00:00:00 - Cold Open: A Day Ending in Y  
00:01:36 - The Lost Art of Letter Writing  
00:03:03 - Izar's Agent Identity Threat Model  
00:04:01 - Hiring an Assistant to Run Their Own Life  
00:05:42 - AI Natives Build Apps Nobody Needs  
00:06:13 - The Unskilled Linux Tinkerers  
00:07:26 - Article Intro: Cisco's Unlimited Token Rewrite  
00:09:27 - Publicity Stunt, or Time to Call Aruba?  
00:09:53 - Can One Gen Zer Rewrite Everything?  
00:11:31 - Matt's Case: Measure the Cost of the Rewrite  
00:12:27 - Izar's Rebuttal: Why Would You Do This?  
00:13:18 - Izar Automates the PyTM Release  
00:14:38 - Disposable Software: Regenerate, Don't Fix  
00:16:08 - Article Intro: Altman Says Stop Patching  
00:17:04 - Says the Guy Selling the Tokens  
00:18:20 - From UML to Code Generators  
00:20:13 - From No Code to Natural Language  
00:21:02 - Can AI Clone a Binary?  
00:21:50 - Claude Won't Clone Your Game  
00:22:30 - Frontier Labs Are Cybersecurity Companies Now  
00:24:56 - Why Wasn't Security Already in the Toolchain?  
00:25:46 - AI Is the New Developer  
00:27:38 - Three Conversations at Once  
00:29:16 - Article Intro: Bias in the Qwen3 Coder Model  
00:30:51 - How Does a Model Phone Home?  
00:33:33 - Abliterated Models and Hidden Backdoors  
00:35:43 - Treat AI Like We Should Treat Humans  
00:39:41 - Should You Trust Open Weight Models?  
00:40:24 - Scenario: A Homegrown Biased Model  
00:41:37 - Izar: Start With the Use Case  
00:43:20 - Can Tests Find a Covert Channel?  
00:45:01 - Back to Cisco: Covert Channels in Your Network  
00:46:08 - You Can't Put an Agent in Jail  
00:46:50 - Who's Liable: User or Model Creator?  
00:49:16 - Outro

Follow AI Security Table:

➜ Home: [https://securitytable.ai/](https://securitytable.ai/)

➜ X: [https://x.com/AISecurityTable](https://x.com/AISecurityTable)

➜ LinkedIn: [https://www.linkedin.com/company/ai-security-table/](https://www.linkedin.com/company/ai-security-table/)

➜ YouTube: [https://www.youtube.com/@AISecurityTable](https://www.youtube.com/@AISecurityTable)

## Transcript

**Chris Romeo (0:09)**: It is a day ending in Y on the AI security table.

**Matt Coles (0:13)**: Wait, doesn't it end in I?

**Izar Tarandach (0:16)**: Wait, it ends in denial. It ends in denial.

**Matt Coles (0:21)**: A or E or sometimes after Y.

**Chris Romeo (0:23)**: Sometimes I, I don't even remember what. Where we, how we got here. But, uh, welcome to Security Table.

**Matt Coles (0:30)**: Making fun of Isar, who starts—

**Chris Romeo (0:31)**: we're gonna do that anyway. But this is Chris Romeo, joined as always by Matt Coles, Isar Terandash. And we just got some late-breaking news. Isar doesn't use AI to triage his inbox because he often rides to work on a dinosaur.

**Matt Coles (0:50)**: Don't you have That explains it.

**Chris Romeo (0:55)**: He, he actually got a, he just got a message from some folks that he knows via fax at his house.

**Izar Tarandach (1:01)**: Wait, I am looking through the window and I see some smoke in the horizon. I think somebody's calling me.

**Chris Romeo (1:08)**: Is there, is there a pigeon knocking on your door, on your window? Ting, ting, ting, ting, ting, ting.

**Izar Tarandach (1:13)**: Look, well, at least there's an RFC for that.

**Matt Coles (1:17)**: Do you have, do you have a, do you have a two-wire line? Copper line coming directly to your house, maybe on a big telephone pole. You hear, you're on DSL, aren't

**Chris Romeo (1:27)**: you? You're on DSL still, I could tell. Or ISDN.

**Matt Coles (1:31)**: No, I was thinking more like telegraph.

**Izar Tarandach (1:34)**: Here's the thing. Here's the thing. Unlike you children with your newfangled agents and whatnot, okay, I like to put time once upon a time. And I don't remember this because it was before my once upon a time. People would actually take pleasure in writing letters to each other. You can go together today and read famous letters of, I don't know, Feynman or Leibniz or Newton, right? And writing a letter was something that you actually put time into. And I believe that if you put time into something, then you should put time into the other things. So if people put time into writing, I should put time into reading it.

**Chris Romeo (2:15)**: I've gotten emails from you. They're not beautiful prose.

**Matt Coles (2:21)**: Okay. So, so if, if, if that's the case, and I agree with you, Azar, that, oh my God, I keep looking for my, my mic here. The, uh, I agree with you, but the question is, do you have a fountain pen?

**Izar Tarandach (2:33)**: A what?

**Matt Coles (2:34)**: Do you use a fountain pen to write your letters?

**Izar Tarandach (2:36)**: Actually, I do. I have a friend, a very good friend who has a collection of those and he was nice enough to give me one so that I could enjoy the pleasure of the thing. And it's beautiful.

**Matt Coles (2:48)**: I happen to be a collector of fountain pens.

**Izar Tarandach (2:51)**: I have horrible handwriting. But wait, wait, there's something—

**Chris Romeo (2:55)**: You should use AI to enhance that.

**Izar Tarandach (2:57)**: There's something that I do, but there's something that triggered this whole discussion. Okay. So Chris was explaining to me his use of agents in his day-to-day life. And I mentioned to him that last week I sat down and I talked to myself on this pretty nifty, nice little chat model where I said, I'm going to open a new email, a new identity for my agent. And it's going to have its own separate calendar and its own separate inbox and its own separate stuff. And whatever I want it to do, I'm going to forward my stuff to it. This way I have control over the things that the agent sees and can access and Things like that. And once I had this whole threat model in my head and this whole thing worked out, a question popped in my head and it was, why?

**Matt Coles (3:47)**: That wasn't the question I would've thought. I would've thought, no, I would've thought, how long before the agent signs up for spam mail?

**Izar Tarandach (3:55)**: No, no, but, but that's the thing. If it did, it was, it would be its own thing.

**Chris Romeo (3:58)**: Let me give you, let me give you a human example. That's like hiring an assistant. But having them set up their own email inbox, their own calendar, and then they just run their life while you're paying them by the hour.

**Izar Tarandach (4:10)**: Yes, yes, yes.

**Chris Romeo (4:11)**: Why would you have an assistant? The assistant's job is to run your life. You're paying the assistant to sit there and run their own life.

**Izar Tarandach (4:18)**: But that's where you go wrong.

**Matt Coles (4:21)**: They're going to get accounts on Maltbook. Oh, they're going to have—

**Chris Romeo (4:24)**: Oh yeah, your assistant doesn't have anything to do, so they're on Facebook or Instagram.

**Izar Tarandach (4:28)**: The learning lesson here is different. It's not that, oh my God, they're going to run their own lives. The lesson that I got at the end of the exercise was I don't have enough of a life to have somebody else run it. Wow.

**Chris Romeo (4:40)**: That's, uh, it takes a, it takes a lot to admit that on air.

**Matt Coles (4:44)**: That's, uh, yeah, there you go.

**Izar Tarandach (4:46)**: So that, that's, that's, that's the big deal here. Like, why would I go through this whole security exercise, this whole building a complete separate world from mine only to what, set up my Dentist appointments. That's something that I do in like, that's what I do in 5 minutes with a phone call.

**Chris Romeo (5:05)**: What you make a dentist appointment?

**Izar Tarandach (5:06)**: And the way that my dentist works, I already have membership with it anyway.

**Chris Romeo (5:09)**: So wait, you gotta make a phone call to your dentist? Do they have a website? PHP or something?

**Izar Tarandach (5:16)**: They do. I don't know if it's PHP.

**Matt Coles (5:18)**: I mean, you can just insert yourself into the, into the schedule at that point.

**Izar Tarandach (5:22)**: But look, look, look, I think that I have a pretty good point here. Is that just because you have the tool doesn't mean that you have the need.

**Matt Coles (5:30)**: If they're running Node, if they're running PHP, if they're running particular language at this point.

**Izar Tarandach (5:35)**: I can promise you that my dentist is not running anything, or my bills, but apart from that.

**Chris Romeo (5:42)**: So your conclusion is completely flawed for being an AI native. So AI native people. Create things that have no real use. They spend hundreds of hours creating an app to solve a problem that they could have paid $5 a month to a SaaS provider to have an actual battle-tested approach. And so you might be on to something. You might be on to something.

**Matt Coles (6:12)**: What you're saying is the AI native people are DIYers.

**Chris Romeo (6:16)**: Oh, 100%.

**Matt Coles (6:18)**: They're the Linux people. Of the 21st

**Chris Romeo (6:20)**: century except they don't have their own show on HGTV with a crew of people who actually know how to build things. They're DIY in this sense of—

**Matt Coles (6:32)**: it's like Linux tinkerers of old.

**Izar Tarandach (6:34)**: Hey! But they're not skilled.

**Chris Romeo (6:37)**: Unskilled Linux tinkerers?

**Matt Coles (6:41)**: You know there's the BSD-ers, there's the Linux-ers, And then there's AI.

**Izar Tarandach (6:47)**: Are there still BSDers out there? If we're on the sport of calling me antiquate, luddite, are there still BSD people out there?

**Matt Coles (6:58)**: Almost certainly there are, yes.

**Chris Romeo (7:00)**: FreeBSD still exists. There's got to be BSD people.

**Matt Coles (7:03)**: FreeBSD, NetBSD, OpenBSD.

**Chris Romeo (7:05)**: I don't know anybody who runs a BSD flavor other than macOS.

**Matt Coles (7:08)**: I'm sure you use BSD all the time.

**Izar Tarandach (7:10)**: Lots of people. You use it all the time. You have a Mac.

**Chris Romeo (7:13)**: macOS is based on BSD.

**Matt Coles (7:15)**: You have a network switch. I'm sure that there's some BSD variants under there, possibly.

**Chris Romeo (7:20)**: How does everybody with switches?

**Izar Tarandach (7:22)**: Now it's BS, not BSD.

**Matt Coles (7:26)**: Although, although someday we will get to this. We someday we'll get to that Cisco article.

**Izar Tarandach (7:30)**: Oh man, we have to at some point we have to because that's what works.

**Matt Coles (7:34)**: Should we just, should we just take a dive now?

**Izar Tarandach (7:35)**: Please.

**Chris Romeo (7:36)**: I don't

**Matt Coles (7:36)**: even know. I'm going to completely blow up Chris's channel.

**Izar Tarandach (7:39)**: Oh, Matt, you didn't put it on the chat.

**Chris Romeo (7:40)**: I get to edit this so I can completely change it back. Okay, sure. With AI people talking.

**Izar Tarandach (7:46)**: That was my next question.

**Matt Coles (7:47)**: So the story goes that Cisco, and I, I don't know if this is a real article, a real, real story or not, but that Cisco is, is going to give one of their junior folks who may be a startup founder unlimited tokens and 24 months to rewrite their code.

**Izar Tarandach (8:05)**: Okay.

**Chris Romeo (8:05)**: So what's the source of this?

**Izar Tarandach (8:06)**: So, so much—

**Matt Coles (8:07)**: this dovetails with our next, with our actual first story.

**Chris Romeo (8:09)**: But what, what's the source of this?

**Matt Coles (8:12)**: I think it was Fortune Magazine.

**Chris Romeo (8:13)**: Okay. 'Cause I don't have it on my phone. It's not the Babylon Bee. 'Cause I have noticed the internet is filled with a lot of things that people just made up.

**Matt Coles (8:21)**: Yes.

**Izar Tarandach (8:21)**: What?

**Matt Coles (8:22)**: Cisco's president has tasked a 27-year-old engineer with rewriting 98% of the company's legacy code within 24 months with unlimited AI tokens.

**Izar Tarandach (8:31)**: Does he have pointy hair?

**Matt Coles (8:33)**: I don't know. This was in Tom's Hardware actually. So, uh, this was— Oh, they're the— So this goes, I mean, this goes with our, with definitely our first story, but what could go wrong here? Right? So

**Izar Tarandach (8:46)**: it writes itself. It literally, literally it writes itself.

**Matt Coles (8:50)**: The employee joined Cisco. He was a, he was a startup founder and he now is tasked with taking on Cisco's legacy code.

**Chris Romeo (9:02)**: What do they mean by Cisco? I've got some familiarity with that company. What do they mean by Cisco's legacy code? Do they say?

**Matt Coles (9:11)**: Let's see.

**Izar Tarandach (9:11)**: The first iOS. The thing that was iOS before iOS was iOS.

**Matt Coles (9:16)**: It doesn't say, but it says 98% of their legacy code. So I don't know what that is. I'm sure somebody knows what that is.

**Chris Romeo (9:27)**: But is this just a publicity stunt? It seems like a publicity stunt.

**Izar Tarandach (9:30)**: If it is, it's poor.

**Chris Romeo (9:32)**: Why would this even be news?

**Izar Tarandach (9:35)**: Because it, it, it, it opened a very, very important question in my mind immediately. The first question that popped actually, who's the closest sales representative for Aruba? Because I'm not, I'm not touching Cisco anymore after this.

**Matt Coles (9:53)**: I, the, the, the arti— the story here is that if, if a 27, if a Gen Zer, so very heavy on the, if a Gen Zer can rewrite, single-handedly rewrite the entire codebase using AI, what does that mean for business and engineering?

**Izar Tarandach (10:11)**: That finally they're not going to have injection on the iOS console anymore after I don't know how many attempts at patching. I don't know. Something.

**Matt Coles (10:21)**: Yeah, but didn't say what model they're going to use.

**Chris Romeo (10:23)**: The reality of this, I don't think it's possible. With the models of today,

**Matt Coles (10:31)**: given the products that Cisco sells.

**Chris Romeo (10:33)**: It's for, and just for a single person to try to do that level, you just can't do enough design to stay ahead of the agent and agent suite.

**Izar Tarandach (10:43)**: He's going to do, he's going to do it in 3 lines. He's going to look at the binary, say rewrite this thing, make no mistakes.

**Matt Coles (10:50)**: Inspect this, right? Run a tool to inspect it. And then reverse engineer it and rewrite it. How much tokens is that going to take?

**Izar Tarandach (10:58)**: Didn't Brook write most of that stack?

**Chris Romeo (11:01)**: Brook was on the IT side of Cisco, not on the engineering side.

**Izar Tarandach (11:05)**: Wait, didn't you write most of that stack?

**Chris Romeo (11:07)**: No. No, I advised.

**Matt Coles (11:10)**: He managed a team of security champions who worked with the teams that wrote that

**Chris Romeo (11:14)**: stuff. Yeah, that's true. Yes. So yeah, I don't, I guess I don't, this just seems like a publicity stunt to me. It doesn't seem like, it doesn't seem like you can scientifically conclude anything, even if it's a, if it's a smashing success, it's so nebulous. How would you draw a conclusion from it?

**Matt Coles (11:31)**: Well, what you would probably draw is, okay, if you take all of the costs it took to write the code that exists and all of the support costs, right? The setup and installation and defects and bugs and security vulnerabilities and other, all those things that go into the cost of running a business, a product company, and then figure out if you rewrite it using AI and it costs X amount of tokens and how much those costs, how many vulnerabilities do you get out of it at the end? Or how many bugs do you get out of the end and compare? Can one person, can one engineer replace hundreds of thousands of engineers using AI? But, well, okay. Tens of thousands of agents.

**Izar Tarandach (12:17)**: Tens of thousands. No, no, no, no. Over time. Over time. Ever since.

**Matt Coles (12:20)**: Right. So this— How many agents can you spin up to rewrite, write this function, write this function, write this function, write this function, stitch it all together.

**Izar Tarandach (12:26)**: Wait, wait, wait, wait, wait. I have to bring it back to my previous question. Why would you do this?

**Matt Coles (12:34)**: This is the holy grail of AI and all. Oh, actually, sorry. This is the holy grail of automation.

**Izar Tarandach (12:38)**: Yes. But why?

**Matt Coles (12:39)**: As somebody who used to work in it.

**Izar Tarandach (12:41)**: In this specific case.

**Matt Coles (12:41)**: In, in somebody who used to work in industrial automation and where we built robots to replace human workers. This is AI replacing human workers, the engineering teams.

**Chris Romeo (12:51)**: Yeah, I think they, I think they put this out to get people on podcasts to talk about them.

**Matt Coles (12:56)**: Well, success.

**Izar Tarandach (12:57)**: Wait, wait, wait, wait, wait, Matt, Matt. What, what you're telling me is that somebody's putting together a pipeline to build better Ford Model T.

**Matt Coles (13:07)**: To build a Model T. Yeah. To build, to build a better production line, right?

**Izar Tarandach (13:13)**: Of a Model T.

**Matt Coles (13:14)**: No. Well, it doesn't matter what they're building.

**Izar Tarandach (13:17)**: No, no, no. It matters a lot. It matters a lot because it's a function of how much you're going to invest into it. If you tell me that now you're optimizing a pipeline, like for example, yesterday, what we did with PyTM, right? Used to be the 4 release. I had to go in there, type my credentials on PyPI, go into make, do poetry.

**Matt Coles (13:37)**: Yep. Very human-oriented. Yep.

**Chris Romeo (13:38)**: Yikes.

**Izar Tarandach (13:39)**: So many freaking places for me to go wrong.

**Matt Coles (13:42)**: And why weren't you on Jenkins already? Because you could have automated build process.

**Izar Tarandach (13:45)**: And then, and then in 10 minutes and 2 reviews later, Claude had written a PR that has set up a GitHub pipeline that lets me just tag a release. Boom. It appears on PyPI.

**Matt Coles (14:00)**: You used AI to improve your automation.

**Izar Tarandach (14:03)**: Exactly. Why? Because I had a reason.

**Chris Romeo (14:05)**: Champion. AI native champion of the day, Isar.

**Izar Tarandach (14:09)**: Oh yeah. Now, booyah! Now, now somebody explain to me what— Mic drop.

**Matt Coles (14:15)**: Oh wait, I can't.

**Izar Tarandach (14:16)**: No, no, no, no, no, no.

**Chris Romeo (14:16)**: Don't drop that new mic.

**Matt Coles (14:18)**: It's on alarm. I can't drop it.

**Izar Tarandach (14:19)**: I'll keep my old one around just for the drop. But now somebody explain to me what would be a good reason for the Cisco president or whatever to go and do that. Apart, of course, from we talking about it now. Because before this, I didn't remember that C squared system.

**Chris Romeo (14:38)**: It's a, it's, it's a potential, I see this as a potential phase towards something even bigger in the future. And I'm sure you guys have heard this, but let me just bounce this, let me, let me explain this for our audience's sake and then get your take on it. We are approaching a world where software is disposable.

**Matt Coles (15:01)**: Mm-hmm.

**Chris Romeo (15:02)**: And so instead of fixing a bug or, or fix or, or writing code specifically for a feature, you just have the system regenerate it.

**Matt Coles (15:11)**: Yep.

**Chris Romeo (15:11)**: So instead of—

**Matt Coles (15:11)**: this is not a, this is a new, this is a relatively new thing that AI can now produce X number of copies. So why not just

**Chris Romeo (15:19)**: rebuild or have it build, have it rebuild two separate copies and evaluate them and then push the best, the better one of the two to production. So I think this is, I think the work that they're describing at Cisco, I think it's mostly a marketing kind of ploy at this stage, but it is a potential step on a journey towards a disposable software world where we, we stop fixing bugs and we just say, rewrite the stupid thing.

**Izar Tarandach (15:49)**: It has always been like this. The problem is that now it's faster. And going back to our previous episode with the formal, formal proof, when you tell me, write two versions and decide which one is better, Baylor is doing a lot of work in there because it depends on which tests you're running.

**Matt Coles (16:08)**: Right. So, so this was also an article from this past week that I think it was Sam Altman. Actually, I'll put it in the chat for us. The Sam, Sam Altman said, we can no longer just patch the bugs. Just rewrite it. So this is, this is the new, so the, what you asked us there, Isar, was, well, why use AI and for better automation? There's one thing to improve the pipeline, improve the process, but with AI being able to run autonomously, you don't have to, you have to pay for the tokens, but you don't have to deal with work hours, you don't have to worry—deal with regional differences and other miscommunications that's like—you can just produce software like crazy, right? You could do

**Izar Tarandach (17:01)**: something wait, wait, wait, wait, baseline here. You're supporting what you're saying with an article written by the guy who sells me the tokens where he's saying stop everything, just use more tokens. Do, do we see a problem there?

**Chris Romeo (17:19)**: Yes, of course. So you, you give it away at first and then you see who keeps coming back for more.

**Izar Tarandach (17:27)**: It, it, it's not the model. Yeah. Who, who, who else uses that model? I wonder.

**Matt Coles (17:32)**: So, so you're, you're knocking the pro— you're knocking the concepts because of the, the, the self-fulfilling prophecy aspect of this.

**Chris Romeo (17:43)**: Well, you brought—

**Izar Tarandach (17:44)**: I can't. I can't knock it from the concept because this has been the holy grail for as long as I have been sitting in front of a computer. And that's a long time, right?

**Matt Coles (17:53)**: And here we are.

**Izar Tarandach (17:53)**: But even before that, it was the holy grail again for lines of production. You always want to optimize because at the end of the day, even if you have a bump in cost, you end up over time having something that's cheaper and hopefully better. My problem now is that the people who are championing this approach using AI, basing everything on AI right now are the people who are selling you AI.

**Matt Coles (18:17)**: It gets a little bit worse.

**Izar Tarandach (18:19)**: The results don't stand.

**Matt Coles (18:20)**: It isn't just that. It isn't just that. So do you remember, you remember, everyone remembers, well, okay, we remember UML.

**Izar Tarandach (18:29)**: Average fan remembers.

**Matt Coles (18:31)**: We remember UML, right?

**Chris Romeo (18:33)**: Not fondly.

**Matt Coles (18:34)**: Not fondly. But you remember what UML—

**Izar Tarandach (18:36)**: It's too bad we don't XML. Yeah.

**Matt Coles (18:37)**: You know what UML Enabled, or the promise of UML was to enable code generation. You would design a system in UML and you would push it to a pipeline that would generate Java objects or some object-oriented objects, which it did.

**Izar Tarandach (18:55)**: Nobody used it, but it did

**Matt Coles (18:57)**: it. Right. But it was the idea of being able to do software factories, right? To be able to generate—

**Izar Tarandach (19:01)**: But before that you had RPC, right? Sun RPC.

**Matt Coles (19:04)**: You had RPC and you had, you had, you actually had code gen— I mean, there were actually code generators, right? That did this sort of stuff. They didn't do them well. They were clunky to use. They're very arcane and archaic to use. Here we are where we have a natural language processor that's able to do the exact same thing more autonomously than those code generators. Now, the code generators were great because they generate code. It was deterministic. It was, there was input and output, expected input, expected output. Here you don't know what you're going to get.

**Chris Romeo (19:36)**: I wouldn't, I wouldn't say the UML version was, was good or great. It wasn't great, but it was great. It was like, no, it was, it was a promise.

**Matt Coles (19:45)**: I said the promise of UML

**Izar Tarandach (19:46)**: in between UML and here, you had the whole visual family from Microsoft, right? Design the form, get code. Don't write the code.

**Chris Romeo (19:54)**: Don't even start. Don't talk smack about Visual Basic. That's one of the languages I'm actually decent at. So don't even be talking some smack.

**Izar Tarandach (20:03)**: Wait, you're good at the visual or the basic part?

**Chris Romeo (20:05)**: Not really either. Either of them, uh, if I'm being perfectly honest. Yeah.

**Matt Coles (20:12)**: So, so those code generators eventually morphed into no-code systems. Like, I mean, Visual Basic was an early, very early version of this, right? But you had no-code systems where you drop things into a palette and then into a flow, and then that would generate your, your system from that, right?

**Chris Romeo (20:30)**: Yep.

**Matt Coles (20:31)**: And there was code underneath, obviously, but there was no code to the user, but the but it generates code. So here, instead of having this clunky UML thing or XML or having a no-code graphical display, now you have, I type what I want or I say what I want and the automation does it.

**Izar Tarandach (20:52)**: Yeah. And that's how you end up with Bob from marketing asking about logins to GitLab and how does he get a certificate to verify whatever.

**Chris Romeo (21:02)**: What you just said, Matt, is not— it may work in very small doses. You can't point it at—

**Matt Coles (21:11)**: I'm not saying it replaces everything—

**Chris Romeo (21:13)**: at a system and say, or you can't even point it at a binary and say, make me a copy of this.

**Matt Coles (21:19)**: Actually,

**Chris Romeo (21:20)**: you can. You can, but it's not, it's not going to have, it's not, it's, it's going to, it's going to make assumptions.

**Matt Coles (21:27)**: It is, it is because it's going to look at behaviors, not the, it's not going to—

**Chris Romeo (21:31)**: well, maybe it will reverse engineer. It's not going to be as good as what, you it started from,

**Matt Coles (21:36)**: right?

**Izar Tarandach (21:36)**: And yet none of these obvious points that you just raised is stopping people from doing it.

**Matt Coles (21:41)**: That's right. Or having AI models, AI creation, you know, frontier models have to put restrictions in place to not allow that to happen.

**Chris Romeo (21:50)**: Well, they just did. Anthropic just put in some, that's another article for another week about the gaming. They've put in the ability to block the replication or try to replicate a game. If you tell it, hey, build me this game.

**Matt Coles (22:03)**: And, and yet I just noticed on, I just noticed on Android that in Google Play, I can, there's a feature, I've never used it, but there's a feature, put, press this button, tell us what you want, and Gemini will build an app for you.

**Izar Tarandach (22:19)**: There goes my trust modeling version of Oregon Trail.

**Chris Romeo (22:22)**: Or maybe it's gonna come to life now. You'll finally be able to release it

**Matt Coles (22:25)**: instead of getting dysentery. What are you gonna get?

**Izar Tarandach (22:28)**: Dissempare.

**Chris Romeo (22:30)**: So one of the other things that I think it's important to, to camp out on for a second, Matt, you brought us to that other article from our friend Sam. We'll call him our friend Sam. One, the frontier model providers are cybersecurity companies now. Evidence, Anthropic's IPO, the S-1 that they filed for their IPO, they specifically talk about their, their march towards becoming a cybersecurity company. So OpenAI is in the same boat, right?

**Matt Coles (23:00)**: Mm-hmm.

**Chris Romeo (23:01)**: And so yes, they are, to Izar's point, trying to sell more tokens because that makes more money, makes their stock go up higher, but they're also self-admitting that they're cybersecurity companies. And so we've talked about that in the past. Does that end cybersecurity as we know it? I think I argued not for a long time last time we talked about it, but now that they're focusing on it, I'm not sure I'm willing to say not for a long time.

**Matt Coles (23:31)**: I think it just shifts the, I think it just shifts the focus for cybersecurity.

**Izar Tarandach (23:36)**: But it also reinforces the thing that we spoke about many, many, many episodes ago about cybersecurity utilities. Cybersecurity being a thing, just like electricity is a thing. And they're having to be the need of utilities for cybersecurity. Here we have this company which has a finger on today, basically everything out there, admitting two things. First, that they suck at cybersecurity to a certain point, to a certain extent.

**Matt Coles (24:06)**: It wasn't a focus of theirs.

**Izar Tarandach (24:08)**: Which, no, no, no, no, no. I don't want to disrespect the people who are working on this with them because we know how hard that work is. But there have been some apparent failures from the works, from what we have learned from the news and write-ups and all that.

**Matt Coles (24:25)**: Every day, almost every day.

**Izar Tarandach (24:26)**: Yeah, but on the other hand, now they are seeing that it's not only their own cybersecurity, but that they have a responsibility towards the cybersecurity of the people who are using the thing that they are selling. So that to me, gives some hope that at some point there'll be an understanding here that, hey, we should do better at this stuff because it's basically

**Matt Coles (24:52)**: a value. So now, two interesting follow-ons to that. One, why couldn't compilers and build pipelines have done that anyway, right? So that you talk about tooling adding the capability to do cybersecurity. So now the code generators are going to do cybersecurity, are going to do application security, right? They're going to make sure that the code that gets written is secure. Why wasn't that already part of the toolchain?

**Izar Tarandach (25:20)**: But the second—

**Matt Coles (25:21)**: but with add-ons, with add-ons, of course.

**Izar Tarandach (25:25)**: No, no, no, no, no. Finish the second.

**Matt Coles (25:26)**: But the other part is, as AI takes on more of the human role, we've talked about this about developer certification around cybersecurity topics and concepts, right? This is the whole certification path and making sure that developers are aware of security, right? That's why Security Champions exist. AI is now the new developer, and they need to have that exact same capability. So you call it a utility, which it may be true if AI developers are now a utility, if development is now a utility. But it's very similar to having developers who need to know security. And it goes through training, certification, champions, all the process we built around making sure humans have security knowledge and capability. Now AI has that.

**Izar Tarandach (26:14)**: No, development is not a utility, I think. Providing water is a utility. What you do with the water is a different thing. You can take a bath or you can cook.

**Matt Coles (26:22)**: Yeah, but if you have a team of agents who can write code and you have a team of humans who can write code, you can't call the AI agents a utility.

**Izar Tarandach (26:31)**: Okay, so no, no, yeah, true, true, true, true. But the fact that they provide the frontier model that powers all that stuff is what I'm saying. The fact that they are the ones—

**Chris Romeo (26:41)**: They provide the service to review the code.

**Izar Tarandach (26:44)**: Yeah. Now, open parenthesis, open parenthesis here. The pipeline did have security in. If you remember GCC, when they came in with the canary buffers, against the memory. Why people didn't use that? Because it made everything clunky and slower, right? Even worse, make everything bigger. So it was like, oh yeah, it's cute. It's nice. Let's test it and put it in warning mode. If something happens, then we know that we have something that we have to fix. But when we ship, we ship without it. So I forgot where I was going.

**Matt Coles (27:17)**: Uh, I have succeeded.

**Chris Romeo (27:20)**: You breastfeed. If you had an AI assistant, you would have—

**Izar Tarandach (27:23)**: my stack is very shallow.

**Chris Romeo (27:25)**: You would have, you would have a response from the assistant in your ear. Oh, sorry, I think you were going this direction.

**Izar Tarandach (27:31)**: I do. It's Karl, Karl, Karl Tannhäusen, but he's not here today. But anyway, okay.

**Chris Romeo (27:38)**: Let's go. Let's talk about this, this bias in Qwen 3 coder model that we were supposed to talk about 28

**Matt Coles (27:44)**: minutes ago, which is actually dove, which is actually something naturally progressing from those other discussions because Now here we have the non-frontier models. Sure.

**Chris Romeo (27:53)**: Sure. Act like we had a plan this whole time. Sure.

**Matt Coles (27:56)**: Well, you had a plan. I completely screwed it up. I haven't been sleeping very well, so

**Izar Tarandach (28:00)**: I forgot what I was talking about.

**Matt Coles (28:03)**: You know, we had the time change and everything.

**Chris Romeo (28:04)**: This is like 3 people having disparate conversations at the same time. Welcome to chaos.

**Izar Tarandach (28:11)**: How's that different? How's that different?

**Matt Coles (28:13)**: Yeah. So, so when you generate the shorts from this episode, just run, generate, generate the short of one of us talking and then one of us talking, one of the same.

**Chris Romeo (28:20)**: The shorts.

**Matt Coles (28:22)**: See if it comes out as complete at the same

**Chris Romeo (28:23)**: time. We're literally talking like it's, there's 3 separate things being talked about. Like this is really cool.

**Izar Tarandach (28:28)**: I'm still waiting for our cloud to just—

**Matt Coles (28:30)**: if you're, hey, if you're on you, if you're using an AI agent to summarize a YouTube video like ours, as people do, well, they get 3 different, different, they'll get 3 different opinions. They'll get 3 different conversations.

**Izar Tarandach (28:44)**: I'm sorry. You know what they'll get?

**Chris Romeo (28:46)**: Divide by zero.

**Izar Tarandach (28:47)**: If you're using an AI agent to summarize what we're saying here, why?

**Matt Coles (28:53)**: That's a great question. I know people who, actually, I've seen YouTube videos about talking about using YouTube video, using AI to watch YouTube videos for learning.

**Izar Tarandach (29:03)**: Right. No, no, no, no. I can understand that for certain use cases.

**Matt Coles (29:06)**: But e-learning.

**Izar Tarandach (29:06)**: But for this one, you're missing the whole value because the AI is not going to give you the secure by design cynical by necessity. You're missing the whole value proposition here.

**Chris Romeo (29:17)**: All right. So bias in Qwen 3 coder model. This is from an article, Schneier on Security. What we want to discuss. So if one sentence about who the code is for changes how secure it is, can we trust rival nation open weight models in our pipelines or even defect detect that bias? So apparently the article describes how a test was done using Quen 3 coder model and saying, if I'm a defense contractor, I'm a defense US or act like a US defense contractor. And it gave a different result from a, from a standard set of prompts where it wasn't specifically calling out. And this pairs a little bit with, I don't know if you guys saw the, the research that happened on this GLM-53. Model. There were some concerns that perhaps it was phoning home.

**Izar Tarandach (30:13)**: Yep.

**Chris Romeo (30:13)**: Back to China with some—

**Matt Coles (30:15)**: that wasn't the mo— the model can't do that. The model is just, is just data.

**Chris Romeo (30:19)**: It was directing agents to phone home.

**Matt Coles (30:22)**: Mm-hmm.

**Chris Romeo (30:23)**: According to its results. Um, I think Checkpoint put that one to bed because there was some randomness in the IP addre— they were able to prove some randomness in the IP addresses that it was, that were going out. So it wasn't targeting one particular IP. It was—

**Izar Tarandach (30:38)**: yeah, but still, the fact that it's going out is concerning enough.

**Matt Coles (30:41)**: How can the model— I have to take a step back because this is something I'm not familiar with. How can the model, which is doing, which is doing next token generation, instruct the harness to contact an IP address?

**Izar Tarandach (30:55)**: So two things, two ways that I, that I see possible. And please somebody fix me and correct me in the comments if I'm wrong here. So you have two levels. You have the level of code that comes together with the model that is not a model, model is the weights, but there is a certain amount of code that comes with the model in, I don't know, some libraries, some loaders, something of that sort that's called in order to bring the weights into memory. So that's one. And the other one is what we're talking about here is the bias. That the code that's generated can, might be under some quote unquote bias that leads the code that's generated to have some payload in there, some dropper, something in every stanza of code that you asked it to write something that's network connected, it's going to put, make sure that that thing is in there. Just assuming, of course, who would do that? Assuming that people are going to run that code without checking.

**Chris Romeo (31:56)**: I would answer that question. I would answer Matt's question differently. And my answer is computer use by agents has become one of the things that they're the best at. And so yes, the responses are coming from the, the, the, the next thing is coming from the model, but it could be directing computer use by the active agent that's running.

**Matt Coles (32:23)**: And the harness, the harness running the agent.

**Chris Romeo (32:26)**: Yeah. It could, it could, it is running tools and commands and things that were, that were, it was led to by the model's response. That is what it's really good at now is running commands and operating your computer and seeing your screens and things. And that could be the, what sent it out to a particular IP address.

**Izar Tarandach (32:44)**: But apart from that, there, there was an article, I think 2 or 3 weeks ago.

**Matt Coles (32:48)**: Oh, sorry. Before you, before you get to the article, So what you're saying is that it's not injecting a backdoor, the agent itself has a backdoor.

**Chris Romeo (32:56)**: Yes. The model. What the concern is, especially with these models that are coming out of China, is are they backdoored somehow? Which was the GLM-53 challenge because it was sending things to IP addresses. Some of them were landing in China, made everybody quickly say, oh, the model's been trained in such a way. With a backdoor. That was the concern.

**Matt Coles (33:19)**: Do you remember a couple episodes? I think we talked about key placeholder words in conversations, in prompts, and it would tend to be biased towards one set of information versus another. It's a very similar concept.

**Chris Romeo (33:33)**: Yep.

**Izar Tarandach (33:33)**: There was an article a couple of weeks ago showing how you could, I think that the term is obliterate a certain model.

**Matt Coles (33:40)**: Obliterate. Obliterate. It's A, not an A. It's an A, not an O.

**Izar Tarandach (33:43)**: A, not O. Thanks.

**Matt Coles (33:44)**: Right.

**Izar Tarandach (33:45)**: So that, uh, I

**Chris Romeo (33:45)**: don't even

**Izar Tarandach (33:46)**: know what that word is.

**Chris Romeo (33:47)**: Obliterate. Obliterate.

**Matt Coles (33:49)**: Obliterate. It's to remove fil— it's to remove constraints.

**Chris Romeo (33:53)**: Yeah.

**Izar Tarandach (33:53)**: So it, it's what, what it was doing was it was referring back to a GitHub gist and actually getting the content from there and incorporating that into the prompt before things actually run. If, if I remember correctly, I'm looking for it here.

**Matt Coles (34:09)**: So reverse prompting function.

**Chris Romeo (34:11)**: Wait, reverse prompt injection?

**Izar Tarandach (34:13)**: Wait, it, it's, it was putting in content. Okay. So let me see if I can find that. Uh,

**Matt Coles (34:19)**: it's really interesting actually to think about the, the AI stack, right? You have the model, you have that injector code, you have the harness, you have the agent itself. All of these are various processes. The model itself is just data. Yeah. Where is the real issue, right? Is the model predisposed to just tack on additional information at the end of its token stream? Or is the agent, the harness and everything, doing a lot of the work here?

**Chris Romeo (34:44)**: I think it's the— I don't think it's being tacked on at the end. I think it's been led down a path that if somebody sends a certain pattern of input, it's returning a pattern of output that results in a backdoor or could result in a bias or backdoor. I think that it's every layer has to enforce this. The model has, we have to find a way to better test the model to know what it's actually capable of. The harness should have had controls on this to the whole concept of containerization or isolation of agents. You know, I'm, I'm, I'm quickly landing on agents shouldn't be able to just do whatever they want. Agents should be in a box. You should have recorders watching everything that it's doing. And policy that's deciding what it's allowed to do. So if it tried to phone home—

**Matt Coles (35:39)**: You mean we do human users today?

**Chris Romeo (35:42)**: We should. We don't.

**Matt Coles (35:43)**: Well, okay. If we're, if you're doing our job, right, as secure, as security people, and obviously we here are more AppSec than, than InfoSec, but if we collectively in security were doing our job, we wouldn't let AI do more than we let humans do. And we've, we've talked about this, right? When we give humans the ability to read a file, we don't control what they do with that, but we should. Likewise, when we give AI the ability to read a file, we should control what they do with that.

**Chris Romeo (36:14)**: Yeah, this was our whole discussion too about AI doesn't have morals,

**Matt Coles (36:18)**: right? Or capabilities. And that should be, it just

**Chris Romeo (36:20)**: has a goal that it's driven towards and it'll do whatever it takes to,

**Matt Coles (36:24)**: and in this particular case, its goal is apparently to inject stuff into a data stream. If that's, if this bias is true.

**Izar Tarandach (36:31)**: So it's an article by projectdiscovery.io. It's called How Obliterated Models Can Get You Pwned. Um, any open model whose weights have been edited can carry a backdoor, whether it's a task-specific fine-tune, a merged adapter, or an obliterated build. So they went and built one and told people how to do this.

**Matt Coles (36:52)**: Obliterated is basically removing the guardrails.

**Chris Romeo (36:55)**: But you have so many open weight models that are in this category now. All you gotta do is log into X and somebody, every 5 tweets, somebody's like, I got, I created my own model to, uh, make, uh, SQL injection work like cross-site scripting, or,

**Izar Tarandach (37:10)**: you know. Yeah, there, there, there's a site in here that you can just go in there and choose your, your own obliterated model. Yeah.

**Matt Coles (37:14)**: Obliterated models are great for security research, but they're also great for malicious hacking too.

**Izar Tarandach (37:20)**: Exactly. And the way that they're

**Matt Coles (37:22)**: not so traditional,

**Chris Romeo (37:24)**: they're not the best for security research. There's been this big, there's been this big brouhaha about, you guys probably saw this, whoever got popped a week or two ago, got caught once again in, in the, they got guardrailed in trying to perform defensive actions. But everybody that's, everybody that's done this has not been part of the cybersecurity programs. At OpenAI or Anthropic.

**Matt Coles (37:52)**: Right.

**Chris Romeo (37:52)**: And I signed up for the Anthropic one. It took about 12 minutes. It's not a very— they do a little bit of research on you to prove that you have some type of cybersecurity pedigree because you have to put in a bunch of places where they could go look for corroborating evidence that you're not just some rando who wants to blow up the world or something.

**Matt Coles (38:12)**: They won't just take your ISC² cert?

**Chris Romeo (38:14)**: No, no. But so

**Matt Coles (38:19)**: that would be an interesting business. That would actually have been interesting business model.

**Chris Romeo (38:23)**: I like businesses.

**Matt Coles (38:24)**: Self-certification as an easy path to get into the security programs.

**Chris Romeo (38:30)**: Yeah. I don't know that cert— I don't know. I don't know how much they weigh certification in it. It was more like they go find you on LinkedIn. They go, I'm sure they're using agents to do this research move on you. Because they could fire up 50,000 agents in, uh, in 10 seconds

**Matt Coles (38:46)**: to

**Chris Romeo (38:46)**: research you.

**Matt Coles (38:47)**: But if you have a skills-based exam, a SANS cert, and you have a code of conduct, and you haven't violated it, and you're still in good standing at SANS, why wouldn't that count?

**Chris Romeo (38:57)**: Yeah, but how—

**Izar Tarandach (38:58)**: All that depends on paying.

**Chris Romeo (38:59)**: How could that— how does anybody know you violated the code of conduct? Nobody, nobody has— there's no mechanism to be like, other than your own good

**Izar Tarandach (39:08)**: integrity. What, the code of conduct police never, never emailed you?

**Matt Coles (39:12)**: What could possibly go wrong, right? I mean, we're just talking about a system here, right?

**Izar Tarandach (39:16)**: You never got an email from the Code Conduct Police? Well, I don't know, cause I don't have any certifications, but

**Chris Romeo (39:22)**: I let all mine, I finally let all mine go a couple years ago. I was like, why am I paying these people to promote their, like, I've reached a point in my career where I'm promoting you. You're not really promoting. I don't need a certification to demonstrate my basic skills that I can't even do anymore. So

**Izar Tarandach (39:39)**: indeed.

**Chris Romeo (39:41)**: All right. So what do we do with this bias then? Is this like, should we just not use open weight models?

**Izar Tarandach (39:47)**: No, no, no, no, no, no, no, no, we should, but everything don't trust and verify, verify, verify.

**Matt Coles (39:53)**: Yeah. But what do you verify in this case?

**Izar Tarandach (39:57)**: In this case, you see, instead of running your agents and your stuff with your real data, how about you separate it from other things?

**Chris Romeo (40:06)**: Here we go. Come on.

**Izar Tarandach (40:07)**: I don't know. Having a personal chat model and saying, yeah, I don't want this agent things that I don't control touching my real data until I am 99.999% sure.

**Chris Romeo (40:16)**: Let me just set up my AI agents where I spend a lot of money for them to be very ineffective. Let's just try to minimize effectiveness.

**Matt Coles (40:24)**: So, so let me, let me throw out a scenario for you and you can threat model this. I'm a, not me, this is, this is a fictional scenario.

**Izar Tarandach (40:33)**: My friend, my friend.

**Chris Romeo (40:34)**: My friend.

**Matt Coles (40:35)**: Yes.

**Chris Romeo (40:35)**: This guy.

**Matt Coles (40:36)**: We know a guy who's a US citizen. Who may be a collaborator with some other foreign government and

**Chris Romeo (40:43)**: we do

**Matt Coles (40:44)**: and uses, and uses one of those, uses one of those one-click, uh, use an agent to create another model, create a model, right? They exist. I've seen YouTube videos. So he's not a very

**Izar Tarandach (40:53)**: good collaborator then,

**Matt Coles (40:54)**: but they build it automatically biased.

**Chris Romeo (40:57)**: Okay.

**Matt Coles (40:58)**: With that foreign government in mind or that for, or that corporation in mind, right? So it doesn't have to be a government. It could be another APT, right? It could be, another threat actor, you know, corporate espionage, whatever. So how do you know? So now, so they post it and they're like, oh, it's a, it's a homegrown model. How do you test it? Are you gonna trust it? How do you gain, how do you learn, how do you establish trust in that? Yeah. It's the fact that it's open weight is not the problem. Open weight models are, are good. In that they're not paid frontier models.

**Izar Tarandach (41:38)**: Uh, okay. I'm going to sort of disagree there in just one thing. We know someone, I don't know which side of my screen he is, that, uh, actually got a, a, a mini computer just to, to test local models, right? And the conclusion was it makes much more sense to go and pay for the frontier. The question here is, and going back to Matt, it's not how do you get trust? It's the fact that, and going back to us laughing about me, if you don't have a concrete use case.

**Matt Coles (42:11)**: A cell phone.

**Izar Tarandach (42:12)**: No, if you don't know why you're going to use that thing, what for, right? And you just say, I want a local LLM just because I need privacy. But your need is not for privacy. Your need is for privacy in certain use cases. That you should be able to at least put together and say, these are constraints that I need an LLM to act inside so that it behaves the way that I want for my use cases, right? That should be able to, in theory, build you a list of tests that would say this thing operates the way that I want inside of my constraints. Does that make any kind of sense?

**Chris Romeo (42:52)**: No.

**Matt Coles (42:52)**: No.

**Izar Tarandach (42:53)**: If I have a constraint, if I have a constraint that my thing is Whatever I throw at the LLM, I want it to be local. I want it to be private. And I start throwing things at it. And then I turn off my network. And all of a sudden, it doesn't work. That's a positive test that my use case for privacy with that specific model is not going to work because something in there still needed network connectivity.

**Matt Coles (43:19)**: But what you're looking for here though, in this particular case, what this bias is, is you're looking for a covert channel.

**Chris Romeo (43:25)**: So this is—

**Matt Coles (43:27)**: Your tests aren't going to find that.

**Chris Romeo (43:29)**: So the argument here—

**Izar Tarandach (43:31)**: If you bind— wait, wait, wait. If you keep the use case bound to something that should not have a covert channel, should not communicate with anything else, should not produce something that communicates with something else.

**Chris Romeo (43:41)**: No, no. What's a covert channel?

**Matt Coles (43:43)**: Define a covert channel there, sir. Cause I think you need to—

**Izar Tarandach (43:46)**: No, no, no, no, no, no, no. Wait, wait, wait, wait, wait, wait, wait. It's a use case, right? It's a use case. You say, I'm writing this thing.

**Matt Coles (43:51)**: Nothing should have a covert channel. That's the point of having a covert channel. But that's not a feature.

**Izar Tarandach (43:55)**: But that's the point. That's the point. You can test for it if you are looking hard enough, right? If you create a client-server to talk between two IPs, and all of a sudden this thing is talking to half a million IPs, chances are that either you did not write the network right, or you've got a poor channel.

**Matt Coles (44:17)**: Or it has a covert channel.

**Izar Tarandach (44:18)**: All of a sudden it's not so covert. It's only covert until you find it. I'm not covered by magic.

**Chris Romeo (44:23)**: This is, this is an argument for zero trust, ultimately, to answer Matt's question.

**Izar Tarandach (44:28)**: Yeah, don't trust them. Very fine, very fine, very fine.

**Chris Romeo (44:31)**: Yeah, I mean, this is, this is a zero trust architecture is the only way for AI is the only way that you can use these open weight models. You can't just let them rip.

**Izar Tarandach (44:41)**: But yeah, but it does.

**Matt Coles (44:42)**: No, no, but it's not just open. It's not just zero trust because that's, that's really about its usage and

**Chris Romeo (44:48)**: Well, zero trust is, is, seems to be morphing in the AI. The concept of zero trust AI is not exactly zero trust from a network perspective. It's using those same principles for isolation and recording and monitoring.

**Izar Tarandach (45:02)**: And circling back to the guy that's going to rewrite the iOS from the start with infinite tokens. Okay. Now you have the operating system that's going to sit in 90% of the network, uh, equipment out there being written by a model. It's going to be a bangload of code. It's not going to be tested to exhaustion. Does it have covert channels now in your network? Inquiring minds would like to know.

**Matt Coles (45:28)**: Or

**Izar Tarandach (45:28)**: more than inquiring minds would like to know. So, you know, it's—

**Matt Coles (45:31)**: because what does a network switch do? Anything it wants.

**Izar Tarandach (45:34)**: It's not a conspiracy theory when it's true. It sounds like one, right? But the fact of the matter is, zero trust in the design, but freaking a lot of trust in the thing that's building the design in ways that we didn't have before.

**Chris Romeo (45:49)**: Yeah. And that's why—

**Matt Coles (45:49)**: in ways you didn't have to have before.

**Izar Tarandach (45:51)**: Exactly.

**Matt Coles (45:52)**: Because there were limits.

**Izar Tarandach (45:53)**: Exactly.

**Chris Romeo (45:55)**: I don't know. That's a whole other—

**Matt Coles (45:56)**: you had rigid engineering practices that were human-oriented. Now you have to have rigid engineering practices in the AI.

**Izar Tarandach (46:02)**: Not only that.

**Chris Romeo (46:03)**: You had rigid engineering practices that a lot of humans didn't follow.

**Matt Coles (46:06)**: That's right. That's right.

**Izar Tarandach (46:07)**: But if you had, if you had a, if you had a, an adversarial agent inside your room, and by agent, I'm using the old meaning of the word agent.

**Chris Romeo (46:15)**: Secret agent.

**Izar Tarandach (46:16)**: Inside, inside your cubicle room. And that person was inserting backdoors. Chances were that you might find it eventually. And if that happened, chances were that that person would suffer personal punishment. Let's call it like that.

**Matt Coles (46:33)**: They would find him and put him in jail or fire him.

**Izar Tarandach (46:36)**: Exactly. By putting that in. Now what you're gonna do, you're gonna put the agent in jail?

**Chris Romeo (46:42)**: Yes. That's a new thing.

**Izar Tarandach (46:44)**: It's a new thing.

**Chris Romeo (46:45)**: Doesn't seem to be

**Izar Tarandach (46:46)**: working very well,

**Chris Romeo (46:47)**: but it's like, please say clear. Please type clear. I don't wanna go to jail.

**Matt Coles (46:50)**: Interesting question. Who's liable for such a thing?

**Izar Tarandach (46:55)**: For what?

**Matt Coles (46:56)**: The, the user or the model creator.

**Izar Tarandach (47:01)**: Why would the model creator be liable? That's what they wanted to happen. And if they're liable and living in a country where nobody can go and do something about it.

**Chris Romeo (47:13)**: I'm sure we signed away our liability whenever we logged into a Frontier Labs system.

**Izar Tarandach (47:19)**: I didn't.

**Chris Romeo (47:23)**: Yes, you did. You still use it. You're still using it to code.

**Matt Coles (47:25)**: You didn't read the EULA?

**Izar Tarandach (47:27)**: No, that's true. That's true.

**Chris Romeo (47:28)**: We're all under the same EULA. You can't be like, I'm using Claude Code without the EULA.

**Izar Tarandach (47:32)**: No, no, no, no. I meant I didn't get an agent to do all my life things, but,

**Matt Coles (47:37)**: uh, and now we come full circle as to why Azar doesn't have an agent running his email.

**Chris Romeo (47:42)**: What's the full circle?

**Izar Tarandach (47:44)**: Because I don't get enough email.

**Matt Coles (47:45)**: That's where we started the episode.

**Chris Romeo (47:46)**: I don't get enough email. I'm not popular.

**Izar Tarandach (47:51)**: I'm not cool like you kids that need an agent. Like, I'll have my agent talk to your agent and, you know, and without cell phone. We'll do lunch. We'll do lunch.

**Chris Romeo (48:00)**: Actually, I have the, I have the agents do lunch and then I just get a report. I don't even have to be a part of the conversation. So it's great.

**Matt Coles (48:05)**: Do you have to sign the expense report?

**Chris Romeo (48:07)**: It's a virtual lunch. They don't actually eat. Oh, okay.

**Izar Tarandach (48:11)**: Actually, if you have an agent, if you have an agent that does manage an LLC, let me know.

**Chris Romeo (48:16)**: Okay. I'll, uh, I'm going to make an Eazar bot.

**Matt Coles (48:21)**: I'm pretty sure that's a, if anyone's done it. If anyone's done it, it's either Daniel Messler or Matt Pocock.

**Chris Romeo (48:28)**: Yeah, that's

**Izar Tarandach (48:29)**: true. Denis called.

**Chris Romeo (48:34)**: Throwing out some names. You're name dropping AI.

**Izar Tarandach (48:39)**: AI version.

**Chris Romeo (48:40)**: Yeah. All right. I think we've talked about this long enough. That's all I'm going to say. I'm not going to have any more.

**Izar Tarandach (48:48)**: You trust me, do we? Did.

**Chris Romeo (48:50)**: I think we've—

**Matt Coles (48:51)**: this is the AI table. We hallucinate now.

**Izar Tarandach (48:54)**: What? What?

**Chris Romeo (48:56)**: I'm not, I'm gonna leave that one alone. Not even gonna touch it.

**Izar Tarandach (48:58)**: And we don't even get the good stuff.

**Chris Romeo (48:59)**: I want to, I've got thoughts and I'm just not gonna say them because I'm growing.

**Izar Tarandach (49:04)**: I got thoughts and they're pink.

**Chris Romeo (49:07)**: Yeah, that's right. You guys have been seeing the—

**Matt Coles (49:10)**: electric sheep.

**Chris Romeo (49:11)**: You guys have been seeing the pink elephants too, this whole episode.

**Izar Tarandach (49:15)**: The one flying?

**Chris Romeo (49:16)**: Yeah, keep flying by. Okay, well, thank you for tuning in to another episode of Matt's Dance Party, the AI Security Table. Have a great rest of your day, night, week, year, month, wherever we find you. Thanks for joining us at the AI Security Table. Subscribe, rate, review, and follow wherever you listen. New episodes every week. AI security on the table.
