# Vibe Coding: Can You Put Your Trust in the Machine?

URL: https://securitytable.ai/vibe-coding-can-you-put-your-trust-in-the-machine/
Published: 2025-07-02
Duration: 2615 seconds
Season: 3
Episode: 9
Hosts: Chris Romeo, Izar Tarandach, Matt Coles

## Show notes and chapters

We’re discussing vibe coding again and how AI-generated code is reshaping software development. We discuss the trustworthiness and maintainability of AI-generated code, examining the challenges of reviewing and integrating automated changes at scale. The conversation spans from practical concerns about code quality to broader implications for open-source projects in an AI-augmented world. We talk about identifying telltale patterns in AI-generated code and why context and traceability are becoming essential for trusting automated systems.

Follow AI Security Table:

➜ Home: [https://securitytable.ai/](https://securitytable.ai/)

➜ X: [https://x.com/AISecurityTable](https://x.com/AISecurityTable)

➜ LinkedIn: [https://www.linkedin.com/company/ai-security-table/](https://www.linkedin.com/company/ai-security-table/)

➜ YouTube: [https://www.youtube.com/@AISecurityTable](https://www.youtube.com/@AISecurityTable)

## Transcript

**Chris Romeo (0:12)**: Hey folks, welcome to another episode of the Security Table. This is Chris Romeo joined by Matt Coles and Izar Terandash. And who knows where we'll go today. It's almost like Maybe we have a plan, maybe we don't, but it will have some huge impact.

**Izar Tarandach (0:29)**: So you had a completely different plan.

**Chris Romeo (0:32)**: Yeah. Now for something completely different, which is a famous line from something, can't remember. But Vibe Coding, we're back on this topic again. And I maybe will start with this Hacker News, not Y Combinator Hacker News, the Hacker News. Have to be specific. It's important in this world. So there was an article published on secure coding in a Vibe coding. Vibe, maybe something like that.

**Matt Coles (1:03)**: I don't know. It was Vibe. Yeah, definitely.

**Chris Romeo (1:05)**: Secure Vibe coding. How the kids are, how the young people are talking now. I'm a little behind the times.

**Izar Tarandach (1:12)**: How them youngsters.

**Chris Romeo (1:15)**: All right. So. This is revolutionary, right? This document. It's just gonna change the world. It's gonna make the AI follow, pay attention and be security conscious.

**Matt Coles (1:29)**: Actually, if you did, if you did say, uh, when you're vibe coding, pay attention to this oracle when, when you're, when you're, when you're creating whatever it is we're creating, maybe. Somebody should try that, actually.

**Chris Romeo (1:46)**: Some of it is, is generated by the, like, you know, multi-stepped prompting is a user or human interaction step, right? It's not a something that the machine can do. I guess unless you have one AI talking to another AI.

**Matt Coles (2:04)**: Well, if you have the— yeah, if you build agents and whatnot, or if you bring in multiple AI, like, hey, when you're if you're talking to Claude to do vibe coding and you want, you know, bring in Gemini, like, Claude, go figure out what Gemini would do and then sort of merge.

**Chris Romeo (2:20)**: So I'm still back on this vibe coding thing is not for production use. We argued about this on a previous episode, but since somebody brought up vibe coding again as something to talk about, like, this isn't for production.

**Matt Coles (2:36)**: People are using it for production.

**Izar Tarandach (2:37)**: Depends on what you call production.

**Chris Romeo (2:40)**: Production is everything and production is nothing. But that's just, they doesn't, that's a nonsensical statement.

**Izar Tarandach (2:50)**: No, but seriously, I mean, I mean, depends, depends on whose standards are. I thought that, I thought that Matt was troubling me there.

**Matt Coles (3:01)**: Sorry.

**Chris Romeo (3:01)**: He's covered his camera.

**Matt Coles (3:03)**: I'm figuring out a better camera position here so I'm not like looking sideways.

**Izar Tarandach (3:07)**: Anyway.

**Matt Coles (3:09)**: This is my better profile.

**Izar Tarandach (3:11)**: But no, really. So the question, the question, one of the questions here is what happens to production nowadays, right? So, it's true. I've seen a lot of people who get to a working MVP and say that this is good enough, this is what's going. And then they wait for some kind of events like, oh, manifest, actually put the everything around. Yeah, a lot of people have had some good success with code that is less production hardened than we usually would think to go into production. Hey, it's not for us to judge. People do what people will do.

**Chris Romeo (3:54)**: So is that code—

**Matt Coles (3:55)**: We can judge.

**Izar Tarandach (3:57)**: Oh yeah, true.

**Chris Romeo (3:58)**: We're security people. That's what we do.

**Izar Tarandach (4:01)**: Yeah.

**Chris Romeo (4:02)**: But is that code maintainable that comes out of the Vibe Coding process right now? Like, let's say, I'll give you a scenario. You use Vibe Coding to create your MVP while you're waiting for some financial event, like an investment. And then you get the investment. Can you then hire 10 developers and they can just pick up that code? Or are they going to have to start from scratch and rebuild?

**Izar Tarandach (4:26)**: So here comes the question. If you're going to hire 10 But why did you call it in the beginning?

**Chris Romeo (4:31)**: Because you didn't have the investment yet. You didn't have the investment, the money to pay for 10. That's right.

**Izar Tarandach (4:39)**: The thing is, I think that we all agree that most chances are that the first one that's coming out is not going to be the final version that's going to do everything that you want.

**Chris Romeo (4:51)**: Sure.

**Izar Tarandach (4:51)**: So ideally it has to be. —make them code. And I've been having a lot of fun lately looking at things that people say that they've coded and looking at the different inputs that they gave to the system and figuring out what kind of impact guardrails put in the, at the vibe time had on the coding time. And you do get a lot of stuff that's— let's— I think, I think, I think my personal opinion is that there is a lot of value in this type coding thing, but not in the ways that most, that a lot of people seem to be thinking. I'm being very careful with my generalizations. Why?

**Chris Romeo (5:48)**: Why now? Why are you being careful now?

**Izar Tarandach (5:50)**: What, what's changed today that, that it's, it's early in the morning on a Sunday. I like, oh, on a Monday. I just don't want to piss off the whole world yet.

**Matt Coles (6:02)**: So why is that any difference?

**Chris Romeo (6:04)**: Why are we, why are we completely changing our dynamic of the situation? Wait, wait.

**Izar Tarandach (6:08)**: I, I was going somewhere with this. Oh yeah. So I, I think that there is some value in, in, in the thing. But for the boilerplate code, right? The stuff that you are more than done writing again and again and again.

**Matt Coles (6:25)**: What's boilerplate in a wide living world?

**Izar Tarandach (6:28)**: You know, the stuff that you would go to GitHub and find the ultimate boilerplate for FastAPI. And that becomes like the the source of your, your own thing, because at the end of the day, you know, it already has the login stuff in there. It already has the recovery email in there. It already has the database stuff in there and all that.

**Matt Coles (6:53)**: Well, so we need to bring in, we need to bring in a new topic. So build versus buy. Oh yeah. And the world of open source, right? So what you're talking about is Well, what's the value of open source in a world of AI and vibe coding? So if in the past you would've used open source as a way to bootstrap your development effort, 'cause, and you would've bought, well, bought, acquired, integrated components, but now why not just let the AI reinvent the wheel every time?

**Izar Tarandach (7:31)**: So that too goes to maintainability, right?

**Matt Coles (7:33)**: Well, it does, but what's the decision of the developer who doesn't know that, right? The developer who's using vibe coding, unless they know, oh, uh, make sure that you make use of open source whenever possible so that you don't reinvent the wheel each time. So that's a really good point.

**Izar Tarandach (7:49)**: And during the weekend, I actually saw a GitHub repository. I think it's called Awesome Windsor Files or something. So Cursor and WinSERP and all that good stuff, you can actually put a series of rules files in place before you start doing your web crawling thing that they sort of like direct the thing around, right? They become part of the context sent in and that influences what you get back. And yeah, some people are having some very good results and some good ideas into what to put on those files. So that it shapes the, the final product. So for example, you could go in there and say, you are always going to use Python and your base framework is going to be FastAPI and you are going to use these libraries and you are going to use Pydantic and all that good stuff.

**Matt Coles (8:41)**: So they're making architectural choices basically for the, for the LLM before, before the live coding session even starts.

**Izar Tarandach (8:49)**: Yeah. And what's in the back of my head is we're seeing here an evolution from best practices and what I would do if it was me writing this thing from scratch. Away from that really, really funny video on YouTube about vibe coding that it's talking to a computer. Man, I laughed so hard at that one. But, uh, If it's moving away from the simple vibe code that we all run around headless chickens a month or so ago saying, oh no, my God, the market is going to be inundated by bad code, right?

**Matt Coles (9:38)**: But, uh, in other words, cooler, cooler, cooler heads prevail and you have proper architecture leading in.

**Izar Tarandach (9:48)**: I don't know if we can say that for sure, because at the end of the day, nobody's out there saying, here's how you vibe code, right? But it does seem to me that there is a movement towards, let's not start from zero, let's start from good foundations. But I don't know how popular that is. I don't know how that reaches the people who are vibe coding their prototypes and then putting them on production and whatnot. No. So I don't know. I guess that it's coming from good people out there who know what good code looks like and are trying to do the right thing. And thank God that we have those. But another thing that I was thinking about and I wanted to run through with you guys is the thing that I'm struggling with writing down. I have a theory that this whole thing is going to, if it hasn't already, and that's where I need you guys' help, that it's going to lead us to parallel of what you see in the SOC or in people's desktops. And that is click-through and alert fatigue. I mean, because of the people that we run with, I can certainly say yes, I know people who read through an EULA before clicking. But most people out there, I would venture, they don't. They just see a box with an OK button, mash on it, and go play Sonic on their Windows box.

**Matt Coles (11:33)**: You haven't given up. On reading the same EULA over and over and over again.

**Izar Tarandach (11:41)**: That's who we are. But even so, even so, I have seen EULAs that have the okay button. But anyway, that's a different discussion. But beyond EULAs, if you have seen or worked with people in a SOC, you do know the amount of alerts they, they get. And sometimes, unfortunately, what can you do? You mash through alerts and mash through messages because your brain already does the pattern recognition and you see that it's a false positive or no, you've seen this thing before and it's not for now, or you do your, your prioritization and that's not something that you need to jump at now. And taking an idea from that model, from those two things, and then putting together the fact that, and I read this article on mental models while developing that I found super interesting, and it was basically saying that Reading code is harder than writing code because when you are reading code, you have to build a model of the system in your head as you go. While when you write code, you have that model in your head and you are translating it into code. So you are bringing it to fruition.

**Matt Coles (13:14)**: Developers have a model in their head of what they're working on. Basically. And— Beyond their, beyond their, beyond their single file? Maybe beyond the module that they're working on?

**Izar Tarandach (13:28)**: Beyond something. They have some kind of structure and they are translating that structure into reality, into code. While when you're reading, if it's not in the context of that thing that you are building, then you have to build those structures in your head and put everything into context. And my God, I'm overloading the word context so much. You have to build that thing as you read to put that code into the right place, into that right structure to decide if this is good code or not. So my contention is that over time, people are going to use agents, live coding and whatever we call it to create code. As Chris pointed at earlier, That code is going to be corrected as it's written. So hopefully it comes out good. And by good, I mean performant, secure, and actually does the thing that it's supposed to do. But I think that as these systems evolve and people probably don't have the money yet to get those 10 developers. Did we talk about the 10 developers here or was it before we started? The recording.

**Chris Romeo (14:39)**: That was, that was about 5 minutes ago.

**Izar Tarandach (14:43)**: Fog, fog. Wait.

**Chris Romeo (14:48)**: This moment brought to you by Yzhar's favorite drink.

**Matt Coles (14:51)**: His Exploding Monster.

**Chris Romeo (14:53)**: Oh, hey, you're not paying us any money. I didn't use the brand.

**Izar Tarandach (14:56)**: Wait, wait, we have to get some. Sorry. Well, good. Monster, I suppose. So, um, monster.com. As I was saying, so people where we use like agents and whatnot instead of those 10 developers to fix the code that was initially written. And when, what they're going to get if they don't just set it for automatic and go home, what they're going to get is a set of PRs. And I think that over time we're going to end up with something like PR fatigue, and people are going to click through PRs assuming that the agent knows what it's doing, which was—

**Chris Romeo (15:37)**: I think that's the scary part, what you just described. And just to summarize, so you said it's more difficult from this point forward to read code than it is to write code. The challenge is we haven't introduced the word trust. Into this conversation yet. And if you can't read the code, if you don't, if you can't process and read the code that's coming out of the machine, then we're in big trouble because you have no idea. You're just, you're just putting your, your trust into the machine and saying, okay, it's, I, I believe that ChatGPT's Codex that I was playing with here while we were talking, or, you know, they all have one. Everybody's got an agent that does this stuff now, right? Yeah. You're inherently trusting that this thing is creating the code in the way you described it— performant, secure. I'll even add focused on privacy of the end user, compliant with laws and rules and regulations of the world. We've got the EU CRA that's dropping now. It's about to flip the whole, the whole, our whole world upside down. Um, but how do you trust, like, how can you trust something that you can't understand what it's giving you? That's the scary thing.

**Matt Coles (16:57)**: Especially as a vibe coder, you don't have a computer science background. You don't have a developer experience, most likely, right? If you're, if you've got, if we have successfully democratized development to a point that anybody can do it, those anybodies may have no experience in actually doing software engineering. Or system design or development. And so how do you trust? Like, what's your basis for trust? Because you don't know how to read the code you're writing in the first place. Like, you couldn't do it by hand, right? It's taking this to an extreme. I know most of the divide coding is probably done by developers who are working on a project, you know, either for an open source project or for a commercial organization. And are using vibe coding because they can be more efficient or more effective.

**Chris Romeo (17:47)**: But I don't know that it's, I don't know that it is that I've heard of, like, it's, it, it, uh, in the market, I was reading something. I don't remember where it was that the marketing teams are some of the earliest vibe coders because they got tired of waiting for, and one of you guys might've said this in the previous episode. That sounds scary. Yeah. They got tired of waiting for developers' priority to catch that for their, their features to be caught up and to cut, to go to the top of the stack. So they started vibe coding their own solutions to their problems that they had.

**Matt Coles (18:21)**: And there, and there, and there's, so let's talk, think about that. Just think about that scenario for a moment, right? So you have a, you have a mark, somebody in marketing who wants, needs something, needs something done because of their customer is, you know, pushing for it.

**Chris Romeo (18:34)**: They have a sale on the line or whatever.

**Matt Coles (18:37)**: So they write code. They don't know what they're getting out, right? They have to rely on others potentially to help them understand or to rely on the agents to be omniscient and be able to produce good quality code, whatever good quality code in their context. But if they get the PRs, they don't know what to do with the PRs.

**Izar Tarandach (19:01)**: Wait, wait, wait, wait, wait. I, I, I have to ask for clarification here. So these marketing people, they, they are starting from zero or are they adding features to an existing system?

**Chris Romeo (19:12)**: I thought what I, what I was reading is they were trying to solve their own business problems that required some development. So they weren't coding features for an, for a product. They were like, hey, I really need this data munged into this format and nobody in engineering or IT is available to help me. So I'm going to use Vibe Coding to write a script. To munge my data from format A to format B.

**Izar Tarandach (19:35)**: Right, because those are two pretty different problems. Yeah, they are. But, uh, just to pull back to my thing, because basically what I'm looking for is feedback before I write it down, and I didn't feel like sending a Google Doc for it to you guys.

**Matt Coles (19:54)**: Well, and you didn't want to ask, you know, what are your favorite LLMs?

**Izar Tarandach (19:58)**: Oh yeah. So, uh, so, uh, so then my question is, do you guys think that this PR fatigue is going to be a thing or not based on what— and do you think the fact that we have tools coming out that declare that they are going to look at my coded code and fix it with PRs, automatic PRs, which may or may not be accepted automatically. Question. Wait, wait, wait, wait, wait, wait. Okay, finish that. Do you think that these tools using different models from the one that actually started with the coding Is that enough to start getting the trust that Chris mentioned? Or are we just making things more complicated than they actually need?

**Matt Coles (21:03)**: Before we answer that, because I think the answer is B, the, how does, what does vibe-coded code look like? How do you know it's Fybe-coded code?

**Izar Tarandach (21:20)**: It has a lot of em dashes. It uses the word Delve as the name of a variable a lot.

**Matt Coles (21:27)**: It has, it has too, it has too many R's. It has too many R's for the length.

**Izar Tarandach (21:31)**: It begins with import in an era of—

**Matt Coles (21:38)**: I'm a little teapot, short and stout.

**Chris Romeo (21:43)**: That's a good question though. Does it have a pattern? I haven't played around with it enough. I'm guessing it does. I mean, does it have, is it, it would seem that it would be more bloated. It probably would.

**Matt Coles (21:56)**: And I guess this goes back to the use versus buy versus build. If you're, if you have, if you don't, I guess there's common tricks and techniques and constructs for a way to write things like loops.

**Izar Tarandach (22:09)**: I'm going to apply the scape hatch of 9 among 10 people who deal with this kind of thing and say, it depends on how the model was trained, man.

**Chris Romeo (22:23)**: You need to put that on a t-shirt. That's a t-shirt.

**Izar Tarandach (22:26)**: I do have a similar one.

**Matt Coles (22:28)**: Even basic things like loop optimization, right? There are ways to optimize loops. Right. And, and there's choices to be made on loop, you know, in the past, at least there used to be choices about loop optimization, which, do you make it more, more verbose so that you could do less, um, you know, compiler optimization, or do you make it more compact and let the compiler figure out the loop? But those sort of structures you're not gonna be able to rely on, I think. Right. There's no, there's no, nothing in, in that will tell you whether it's bytecode or Or not. He'd be relying, you'd be relying on something else to indicate that it's a live feature.

**Izar Tarandach (23:12)**: Um, and I'm sure that there will be telltale. Okay. Because again, because of the way that these things work, right? At the end of the day, it's a bunch of monkeys throwing coins in the air and statistically telling you what the next token is. That's true. There will be patterns. There will be stuff coming out. And again, it depends on how you were trained, right?

**Matt Coles (23:33)**: What patterns do you think? What patterns do you think you would be looking for?

**Izar Tarandach (23:37)**: Just if you could— Very optimized idle loops.

**Chris Romeo (23:43)**: Uh, so I just asked AI this question because why not? We should, of course. Oh, I didn't know that AI was listening.

**Izar Tarandach (23:49)**: Oh, sorry, sorry. We didn't mean it.

**Chris Romeo (23:52)**: It admits very Vibe code reflects what the AI infers from natural language prompts. It may create unconventional structures, inline logic, or variable names that feel right but aren't standardized.

**Izar Tarandach (24:05)**: So if you don't see foo, bar, or baz, it was vibed code.

**Chris Romeo (24:09)**: That's true. We should write a— we should, we should vibe code a tiny script that checks to see if something's been vibe code.

**Izar Tarandach (24:19)**: I don't know, ever since I started drinking Monster years and years and years ago, I always felt like I was vibing like Goldie all the time.

**Matt Coles (24:27)**: But, well, actually, so actually there is a way probably to do this. If you think about, because vibecoding systems are, well, LLMs are trained on preexisting content. If you utilize, and this is, I think what they're doing for um, like college courses and things, you know, papers, if you use an anti-plagiarism system, right, where you're looking for plagiarism, you're looking for pre-constructed pieces of snippets of code. If you see pre-construct— now the problem is if you see pre-constructed code, it could come from Stack Overflow or it could come from GitHub also. So how do you know it's been vibecoded versus developers copying from online sources?

**Izar Tarandach (25:13)**: If it comes from Stack Overflow, the first line always says, you're wrong and you're an idiot.

**Chris Romeo (25:18)**: True. Even on the best possible answer that really focuses on security.

**Izar Tarandach (25:23)**: It's walk away from the computer.

**Chris Romeo (25:27)**: So coming back to your PR fatigue question, I do see that we could enter a world where there is PR fatigue if we've got all these tools. That are vibe coding features for us, and then we've got a separate set of tools that are finding bugs and generating PRs to fix them. We could reach a world where there's hundreds of PRs being generated a day and people just start approving them. They, they, I think it all comes back to that trust problem. What's going to push people to hit the accept button instead of doing a tight code review of it is the trust. And if people trust the output of the AI, or as the trust meter goes up, people are going to click accept PR without looking at it more and more.

**Matt Coles (26:22)**: And— I'm actually, I'm going to take the reverse approach. Accept is the easy answer. And so whether or not you trust the AI, you're going to click accept because as a white coder, And I guess as the bulk of web coders, maybe not developers who are doing web coding, the bulk of them may just, may just trust. There may be implicit trust. And as you learn more about how to make a good informed trust decision, you may not do the click-through because you'll start to, because you'll start to gain the experience to be able to say, oh, that one looks fishy.

**Chris Romeo (27:02)**: Right.

**Izar Tarandach (27:03)**: So let's throw a stone in the pond here. Sure, why not? Who is responsible for identifying the problems that get solved by a model-generated PR? In other words, if we say you need a man, you need a person in the loop to accept the PR. Some of the problems that started the generation of that PR will come from a person that identified a problem and said, this needs solving. Others will come from the tool itself that read the code and said, there's a security problem here. I have to solve this. So assuming that person in the loop is actually a value and, and people will stand for that and do the work. Is it inherently easier to read a PR that solves a problem that a person identified or that a machine identified? How do you know?

**Matt Coles (28:09)**: Or it makes no difference at all. How do you know that a, that a issue, so now we're going back to the issue, not the PR. So how do you know the issue is identified by a human versus not? True. What information do you have that says Bill found this versus LLM Bill?

**Izar Tarandach (28:31)**: Oh, you get that on the PR author. It's going to identify itself as either a person.

**Matt Coles (28:38)**: No. You're really stretching this.

**Izar Tarandach (28:41)**: On the issue. On the issue.

**Matt Coles (28:42)**: On the issue. On the issue.

**Izar Tarandach (28:43)**: Yeah, sure.

**Matt Coles (28:45)**: The report may be a form. It may be a form that gets filled out, right? That gets automatically generated by a service agent. Or by a service. It's just, I mean, it could be a traditional service or it could be an agent.

**Izar Tarandach (29:00)**: It could be a step in the CI/CD, right? Right. Oh, but that's the point. No, no, no, wait. If it's a step in the CI/CD, then yeah, it's going to come from something automatic versus something that is a person who will have an actual user there identifying the issue. Oh, okay. I see.

**Matt Coles (29:18)**: So, although, although AI user, AI agents who are acting as users would be users, right?

**Izar Tarandach (29:27)**: But they would have their own identities. And now we get to the whole non-human identity discussion. That's a great episode for the future. But anyway, so, uh, uh, In my head, it goes again to the mental models, right? If it's somebody that identifies something that needs fixing, chances are that in the course of identifying that, they already built that initial picture in their head of what's wrong, why is it wrong, what should we do about it. And if it's an agent, then of course there isn't, or he won't get it as kind of the PR. So is there a chance here for an advanced fixing agent that not only keeps the PR, but in the comments explain what FPR, what they, I don't want to use the words think or imagine, why they built the PR that way, like what's, chain of thought that led to the PR being that way? Would that solve the problem of, uh, hold on, Dr.

**Chris Romeo (30:42)**: Evil's just entered the chat room.

**Matt Coles (30:55)**: There we go. We got our cat on the internet fixed.

**Chris Romeo (30:59)**: Completely derailed the episode right there.

**Izar Tarandach (31:06)**: So what we're talking about is— I was asking—

**Matt Coles (31:10)**: I think you're doomed one way or another. There's so much— Well, I think there's so much problems in this, right? As agents get better, they're going to become more of a human-like identity. And individuals who are doing vibe coding are not in a position or probably not in a position to understand the differences or appreciate that— not understand, appreciate the differences between human and non-human generated issues and human and non-human generated code changes, PRs. To make a meaningful difference. It depends on what their goal is. If their goal is get this out the door, make a prototype so I can get funding, or so I go off and do something else, or so my friends can play, you know, play some game or something, uh, or whatever, you know, they may or may not care, and they may just say it sucks. Versus those who are doing this for a living and understand how systems work, they're going to take the time regardless of its source. And, and yes, I understand, you know, the SOC example is probably a bad analogy because when you're dealing with a SOC, those are, you may have thousands of alerts streaming at you, right? They're going to be small events, right? They're not an incident declaration. They're events, you know, this file read occurred or this, you know, this port was accessed with this data. A PR is a block of stuff with intent and meaning. And, you know, if you get 1,000 of them, by the way, if I was working on a software project, and let's just say I was working on a mid-sized software project, so I have, there's a few developers on the project, If 1,000 PRs come in, in a, in a group, or even if they come in like one a day for like a regular time period, that may trigger a different response than just simply looking at PRs, right? Because now you have a flood of regular incoming content. What happens?

**Chris Romeo (33:31)**: So we just invented a new category for startups. It's a tool that does PR summarization at scale, and then an agent, the agent decides whether to approve or reject.

**Matt Coles (33:47)**: Can you fine code that?

**Izar Tarandach (33:49)**: I already did it. Somebody already—

**Chris Romeo (33:51)**: It's already available for purchase.

**Izar Tarandach (33:53)**: No, but, but really, uh, I mean, I wouldn't be surprised if one of the, the many correctors, let's call it like that. Out there doesn't already have the functionality, right? But I'm thinking about the three of us have a lot of experience, not only doing code reviews, but watching people do code reviews. How many times did you see someone look at code and say, that's good, move on? When you were like, wait. That's a bunch of loops doing arithmetic on pointers. Perhaps we should sit down and really read that code line by line. How many times have you seen that? And isn't that a kind of fatigue? No. Am I identifying something that already exists?

**Matt Coles (34:48)**: I don't, I don't know if I would classify that as fatigue. I would classify that as hubris.

**Izar Tarandach (34:55)**: Interesting.

**Chris Romeo (34:56)**: That's a good way to describe it.

**Izar Tarandach (34:58)**: Yeah. Interesting. Cockiness.

**Chris Romeo (35:01)**: Oh, I just processed that loop and I understand exactly what it does. Move on. Next. Yep. Yeah.

**Izar Tarandach (35:11)**: Or the same loop. And that's a different developer.

**Matt Coles (35:14)**: That's a different developer at fault, right? Yeah.

**Chris Romeo (35:18)**: I want to come back to something you said, Izar, when you were talking about this advanced agent. Like perhaps we needed an agent that was advanced beyond the other things.

**Izar Tarandach (35:27)**: There we go. He's starting another startup.

**Chris Romeo (35:29)**: Yeah, here we go. But the trace of, like, I would, I would, or I expect traceability of any coding agent that exists today to be a core basic feature, meaning I should be able to understand how it got to the, why it did what it did, and I should have an audit to be able to go back and and say, you're laughing because it doesn't exist, but this is what I demand. No, no, no, no, no, no, no.

**Izar Tarandach (35:54)**: I'm laughing because I think that's adorable that you think it should exist.

**Chris Romeo (35:59)**: It should exist. Of course. You should be able to, you should be able, if we're gonna, if we're gonna rely upon vibe-coded and code, I shouldn't, we should, we should extend the definition to agent-generated code. 'Cause there's a lot more people that are using AI coding tools now that are developers using them, right? That aren't bots.

**Izar Tarandach (36:21)**: I think that the word that they use is 10x.

**Chris Romeo (36:23)**: So 10x accelerated code. There you go. It's, yeah, negative 10x, but that's a different, different conclusion. But look, for this thing to really work, to be able to trust the code that comes out of it, you have to have traceability to be able to understand like what was the prompt that caused it to generate that particular code in the way it did. And if you don't, if you don't have that, then you have, then you have no security architecture of something building with these tools.

**Izar Tarandach (36:55)**: Well, some agents you can point to some piece of code and say, explain to me what's happening here, right? And you are going to get a pretty good explanation to the point that some of it you could cut and paste into a a documentation doc.

**Chris Romeo (37:10)**: But I want the intent of the agent. That's what I want to be able to, as a security person, I want to be able to understand why it broke, why it screwed it up.

**Izar Tarandach (37:21)**: From the little that I understand, and I could be wrong, I'm probably wrong, I don't think that the model even remembers how it got there. It's building that model in its head and forgetting at the same time, building and forgetting, building and forgetting.

**Chris Romeo (37:39)**: No, it's, they're storing everything now.

**Matt Coles (37:41)**: The reasoning and thinking models have memory. Now, how far that memory goes, I mean, you have, you do have problems, I think, with the number of tokens you can store in memory, so you might lose some of that context. But, so actually, Chris, are you talking about Are you talking about what the developer originally asked for? Yes. When coding? Yes. Well, so let's, uh, maybe take a brief moment to talk about that. If you're developing something for a commercial organization, you have a PR, you have a, not an issue, but you have a work, work request, work item. A Jira ticket or something like that, which is why you're doing the vibe coding in the first place as a developer. But otherwise, I don't think you ever get this context, right? If, like, if you're in, if you're looking at open source code, the open source developer had an idea in mind, a concept in mind, and out came code. Regardless of whether anybody bytecode it or not. So even still, you're, you're still, you're still required to, when you're doing the code review, like if you're reviewing open source code and you have an open source project, that project may be thinly described as to what its intent was or what its purpose is, what its purpose is, not necessarily what its intent was, what its purpose is. Right. And, and so you're sort of, stuck with deriving or understanding that meaning from the code and, oh, this looks like it's gonna be used in, I don't know, it's supposed to be implemented in hardware or it's supposed to be for IoT only and not for, you know, commercial applications. I mean, there's a host of things you have to do there to figure that out, right?

**Chris Romeo (39:36)**: What I'm realizing is that the argument that I've been building that I didn't really understand until this moment, until you crystallized it for me, is using an open source project as an example, open source of the future is bigger than just the code. It's the, to be able to trust it, I need to understand, yes, the code will speak for itself, but if the code is so complicated because it was created by AI agents that I can't, that there's things hidden because it used some crazy logic that it thought was optimized to go faster, but it's really hard to read and understand as a human. The context, coming back to Izar's word he's used 100 times in this episode, the context of the, how the code was created becomes part of the open source project and my ability to trust it.

**Izar Tarandach (40:28)**: I don't think so. Just look at what's happening today, right? Developers just download the first thing that Apparently does the thing that they need to happen and happily move on. It could be written in, in, in Fortran with, uh, brainfuck and, and, and whatever. The important thing is that you put in the thing that you have and you get back the thing that you want. I don't think that all of a sudden they're going to start treating it differently because it was written by AI. If it, if at most they're going to think, oh, it's probably better because it was written by AI.

**Matt Coles (41:04)**: Well, that's an inverted, that's an interesting trust model.

**Chris Romeo (41:07)**: Until we have the first named vulnerability as a result of AI agent making, committing a PR to an open source tool.

**Matt Coles (41:18)**: We may have had that already. We don't know that, but that's—

**Chris Romeo (41:20)**: It hasn't been publicized as such. You think it would be, if it happened and somebody thought it, they would have sounded the trumpet because it would have gotten a lot of attention to be—

**Izar Tarandach (41:31)**: No, I'm thinking about that Echo, Echo whatever. Uh, you guys read about that one from last week? No. It was something related to MCP and at the end of the day it was pretty silly but— And it did use prompt injection and Echo something.

**Matt Coles (41:52)**: So maybe this is a call to action to our are dozens of listeners, at least, that if, if you are aware of an AI-committed PR to an open source project that has resulted in a vulnerability appearing somewhere, put the, put it, put the CVE in the comments.

**Izar Tarandach (42:13)**: EchoLeak.

**Chris Romeo (42:15)**: Oh yeah, EchoLeak, I saw that one.

**Izar Tarandach (42:17)**: CVE-2025-32711.

**Matt Coles (42:21)**: All right, we've just—

**Izar Tarandach (42:22)**: CVSS score of 9.3, everybody panic, everybody panic!

**Chris Romeo (42:26)**: Run, run! Wait, come back, come back!

**Matt Coles (42:29)**: 9.3, so network accessible, privileges required, high, high, high.

**Izar Tarandach (42:37)**: What you just saw was Matt's LLM that he has behind his ear, takes the CVSS number and explodes the vector in real time in front of everybody. As you can see, it can be done. Is it useful? No. Inquiring voices want to know. All right.

**Chris Romeo (43:00)**: Well, I think we've had enough fun. This is enough fun for one day. We can, uh, I'm sure vibe coding will be back again within the next couple of weeks. Keeps kind of circling into our view. But for Matt and Isar, I'm Chris Romeo, and thank you for tuning into the Security Table.
