The Evolution Problem: After 100 Episodes, What’s Changed… and What Hasn’t?
With Chris Romeo, Izar Tarandach, and Matt Coles
We made it to 100 episodes, so naturally, we decided to look back and see how wrong we’ve been. In this episode, we revisit some of our past topics, predictions, and hot takes to figure out what still holds up and what didn’t quite land.
About this episode
We made it to 100 episodes, so naturally, we decided to look back and see how wrong we’ve been. In this episode, we revisit some of our past topics, predictions, and hot takes to figure out what still holds up and what didn’t quite land. From “we don’t know what we don’t know” to the evolution of security tools, we reflect on what’s changed, what hasn’t, and why some problems never seem to go away. Along the way, we compare where we were then to where things stand now, calling out a few wins, a few misses, and everything in between. After all this time, are we actually any smarter, or just better at explaining the same problems? This episode is part reflection, part reality check, and a look at what 100 episodes have really taught us.
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 50 min conversation
0:11Chris Romeo Hey folks, welcome to the original Security Table. The one, the only. There's only one on the internet and this is it. You've reached it. This is Chris Romeo, as always, joined by Matt Coles, Izar Tarandash, And to you gentlemen, I'd like to say happy 100th episode. Yeah, we made it. Oh, Matt's been practicing that for weeks.
0:34Matt Coles I can't, I can't do it with two hands. I need another hand.
0:38Chris Romeo Do the too legit to quit, please.
0:40Izar Tarandach Yeah.
0:40Chris Romeo Do the Roman.
0:42Izar Tarandach Oh, you have to say it. It's an M. It's an M.
0:44Matt Coles It's an M. I'm, I'm supposed to, well, I am CA.
0:49Chris Romeo Okay. Wow. We're really, this 100th episode's just going to our heads here.
0:54Izar Tarandach We're, I'll need some more heads for that.
0:56Chris Romeo We're celebrating Reckless with reckless abandon. So, um, the team, we just, we just got back to our houses from Vegas from the 100th episode party. We forgot to invite our audience. Sorry about that.
1:10Matt Coles Well, there was that one guy from Australia, you know, he just happened.
1:12Chris Romeo Yeah, there was a guy, there was a tourist from Australia who just happened to be a listener.
1:17Matt Coles So it's amazing who you bump into in, in Vegas, right?
1:20Chris Romeo Yeah. Remember what happens in Vegas stays in Vegas.
1:24Matt Coles If only, unless what happens on— how many episodes have we talked about this?
1:29Izar Tarandach No, unless you post it on LinkedIn because you're coming back from, uh, we're happy to say none of us has gone to RSA Conf.
1:39Chris Romeo True. So we're in Vegas.
1:41Izar Tarandach That's good. No, that's going to make us the most efficient people in the planet in the next, uh, 3 months because we are not going to be erasing spam from our inboxes all the time.
1:52Matt Coles Oh wait, you don't get spam?
1:55Izar Tarandach Not RSA Conf level spam.
2:00Chris Romeo Yeah.
2:00Izar Tarandach Before it's, are you going to? During is, are you here? And later, do you wanna buy me?
2:08Matt Coles That's true. I got so many dinner invitations for RSA and I didn't know what to tell them.
2:15Izar Tarandach No, seriously, on any given week, if I said, yes to at least like 2 dinner invitations, I would live on steak for the rest of my life.
2:24Chris Romeo Like, hey, take the family with you. Yeah, but like, sure, I'll be there.
2:28Izar Tarandach Actually, somebody sent me an invite for, what was it? Uh, Mad, Mad and Jets, something like that. Basketball game or something. And I'm the only person in my family who does not enjoy basketball. So, okay. Is it too bad if I say, hey, can I not go and send my wife instead?
2:49Matt Coles Take my wife, please.
2:51Izar Tarandach To basketball. Yeah. So what are we doing for the 100?
3:02Chris Romeo Have to buy something in that scenario. Well, we have each chosen an episode from the first 99. That we will then have a big reveal with a lot of drama and discuss how things have changed, how things have stayed the same in regards to the issue that we—
3:25Matt Coles oh wait, I gotta change my answer then.
3:26Chris Romeo 'Cause no, no, you're locked in. It's already, it's in the envelope.
3:31Izar Tarandach I can't do just one.
3:32Chris Romeo All right, well, Izar, you can kick us off then, and then we can always come back to you if we have time for a second.
3:38Izar Tarandach No, no, no, no. You guys go first.
3:41Matt Coles Oh, it's always a way.
3:43Izar Tarandach Mine are usually, it's gonna take a long time.
3:46Chris Romeo Okay. All right, Matt, what do you got?
3:49Matt Coles We don't know what we don't know.
3:52Chris Romeo Very true.
3:52Matt Coles Season 1, episode 18.
3:54Izar Tarandach I don't know.
3:57Chris Romeo I don't know what I don't know.
3:59Matt Coles So I'm looking at the, you know, as we look back over the seasons, I'm looking back at that one. And I don't know why we didn't know this, but certificate pinning.
4:13Chris Romeo Why?
4:13Matt Coles And why certificate pinning? I don't know how we got on that topic, but the title, I guess title is probably the more important part. We don't know what we don't know. And here today, I think we're still in the boat of we don't know what we don't know.
4:29Chris Romeo Right?
4:29Matt Coles We have— what?
4:32Izar Tarandach Isn't that always the case?
4:34Matt Coles Well, I, I hope so. 'Cause if we know, if, all right, we don't wanna dive into AI in this episode as, or as little as possible.
4:42Chris Romeo Come on, dive in. Here we go.
4:44Matt Coles Can't avoid it. We don't know what we don't know, but even worse, AI doesn't know what it doesn't know. And we expect it to know everything.
4:53Izar Tarandach No, AI, AI knows everything that it doesn't know. It just doesn't tell you that it doesn't know.
4:58Chris Romeo You can't ask it the answer.
4:59Matt Coles You can't ask it to tell you what it doesn't know. 'Cause it, yeah. Probably, probably doesn't know that. Or it may not be able to tell you because of filters or because of math.
5:14Chris Romeo But it's okay that we don't know what we don't know.
5:17Matt Coles It would be except for if you're using the tools and it knows and you don't, that's potentially a problem, right? Let's actually, you know what, I'm gonna tie that episode. I don't know why I chose that I chose that sort of at random, but actually maybe not because that Register article we were talking about, when somebody asks, what's a linter? Well, if you don't know what you don't know and you're using a tool that knows and you don't know, then how do you know you did it?
5:50Izar Tarandach I just know that I have a headache right now.
5:53Chris Romeo What?
5:54Matt Coles There's always stuff for us to not know. But if we're relying on a, on something or somebody else to tell us something, how do you trust that?
6:07Chris Romeo You can't.
6:07Izar Tarandach In other words, don't go to college.
6:11Chris Romeo Well, that's a whole separate issue.
6:13Matt Coles That's a whole separate issue.
6:14Izar Tarandach I'm trying to draw a Venn diagram here of what you're talking about. And we have the stuff that we know, we have the stuff that we don't know, we have the stuff that we don't know that we know.
6:24Matt Coles We have the known unknowns, the unknown unknowns.
6:27Izar Tarandach Yeah. I, I, I tried not to go with that particular, well, but, uh, so what you're saying, if I understand you correctly, you can't, you don't, you can't know that. Yeah, I can. No, that's not true. The, the, the name Godot keeps coming up in my head, but, uh, it could be Occam's razor too. Yeah, but wait, wait, wait, let's give people some value here. Okay, so you're saying that if you have a tool and it runs, but you're not quite sure about how to interpret the results, then it's the same as not having the tool?
7:10Matt Coles How do you rely on the outcome if you're not at least familiar with the topic that it's generating? So we have lots of people these days who are doing vibe coding with no computer science software engineering background. So how do you even know that the tool is doing it right? How do you establish that trust?
7:27Izar Tarandach Well, it works. No, no, it's fine.
7:29Chris Romeo You can deploy it to production.
7:31Izar Tarandach No, no, you're missing a very important—
7:33Matt Coles Just because you can doesn't mean you should.
7:35Izar Tarandach You're missing a very important point here. Those people don't care.
7:41Chris Romeo Ah.
7:41Izar Tarandach They care about—
7:42Matt Coles So maybe the episode should have been, we don't know what we don't care about.
7:46Izar Tarandach Yeah, but no, seriously, the point is they don't care what the tools are saying. They care that they have another lamb that they can tell, the tool is saying something that I don't know what it means, fix it. And in many cases, at least in the beginning of the conversation, that will lead to some form of fix. So the point here is that it's not that these people don't know what they don't know. They don't care about knowing it. And I'm going to go back in time the same way when Visual Basic came out, right? And people were running around in circles saying, that's the end of programming because now we can drag and drop everything.
8:26Matt Coles Never mind the VBA.
8:27Izar Tarandach No, no, seriously, if you ever tried to interact with any of the visual suite of Microsoft as it was back in the day, I'm not talking about VS Code today, but as it was back in the day. You know that that was far, far, far away from the end of programming because after you dragged all the forms into place, you actually had to go and do your business logic. And that's usually where people would stop their amazing projects because all of a sudden you had to do it in BASIC. No, all of a sudden you had this strange, very strange window in front of you with lines of gibberish that you were supposed to add gibberish to it. So people were saying that, yeah, it ended and that's it. And we are still here, but sorry, developers are still here. I'm not a developer, but, uh, my point is that nowadays we are much closer to something that writes that gibberish for you. But as anybody who has tried this, right? I recently did something quite big using one of the agents and it took me around 10 days to get it the, the right way.
9:41Chris Romeo And 10 days of straight work.
9:45Izar Tarandach Yeah. A couple of hours every day.
9:48Chris Romeo Yeah. Yeah.
9:49Izar Tarandach But the, the, the, the, the important part is that the return started diminishing very quick. In terms of, oh my God, look at what this thing just did. It would take me 6 months at least to just get to this point. And then it was, okay, so fix this thing. I fixed the thing. No, you didn't. No, no, no, I fixed it. No, no, no, no, no, you didn't. No, no, I fixed it. I'm serious. I'm watching, like, I wrote the source. I fixed the thing. Yeah, I'm watching the result. No, you didn't. Here's a screenshot. You didn't. Oh, you're right. I didn't.
10:23Matt Coles So, It's like talking to another developer, isn't it?
10:28Izar Tarandach It sort of is. It sort of is. But my point is that most people, when they reach that level of, I'm asking you to do something, I'm telling you to do something, I'm ordering, demanding you to do this thing, and the agent is not able, they go into the, okay, This shit doesn't work. Can I find a developer? Right? The, the amount of pruning that you need to do around the edges of whatever it is that you are using the agent for, if it is a serious project, is huge. It's a seriously, it's, it's a big amount of, of conscious work.
11:11Matt Coles But you have, so the thing is though, you have knowledge about what it's, what it does and how it's supposed to do it. And what's supposed to be the outcome, right? So how would you know? So if we think about, we don't know what we don't know, obviously we're talking about security technologies, we're talking about security capabilities or privacy, right? If you weren't already involved in the security space, actually even if you are, 'cause I guess there was some, I mean, there's lots of things we don't know, We don't know. We don't know. Like we don't know when to apply certain technologies, right? And I think that's a universal we, not just us here, but universally in the security space, right? So we can't reasonably ask someone else or something else to fill in those gaps for us. 'Cause what questions do you even ask? I mean, you can dance around the subject. I need something to do X and then have it come up with something. But how would you evaluate that it did the right thing? How would you know that it didn't do the right, have the right outcome? You're able to evaluate it because you have knowledge and experience.
12:32Chris Romeo Yeah. And you're describing a dangerous path that we're on for the future. Where people will build things without understanding how they work.
12:44Matt Coles And not be able to evaluate why they don't work or where they don't work because they may, they may do the one thing they asked for, but not do, or, or do other things, right? The side effects. We've talked about side effects before in this podcast, right?
12:58Izar Tarandach But, but here's the point. A lot of the proponents of the technology, or as I like to call them, pushers.
13:07Matt Coles They say that first one's free.
13:09Izar Tarandach They say that you don't need to understand because the model is going to be so powerful that it's going to do all the understanding for you and you are just going to enjoy the fact that you gave it a mission, walked away, came back and it was ready. Right? The thing is that many times if you look at the behind the scenes of the code that got written and the thing that got deployed, you see that what we, when talking with developers, refer to as mockups, many times for a model, is the code. The, the, the model did something that looks like the thing you asked it to do, but there isn't meat behind it. There, there, there isn't logic behind it. It's just, yeah, it's going to show this and next time it's going to show something else. It's going to look like the right thing. It's basically going to be a coded hallucination.
14:01Chris Romeo Right.
14:02Izar Tarandach It's going to look like the right thing, but it's not the right thing.
14:04Matt Coles And it's what you would get from a new, like potentially, you know, somebody who's an intern, you know, going through a computer science program or learning to become a developer, right? They're gonna make something functional, but it may not meet all the goals. 'Cause you won't know. It may have other side effects.
14:23Chris Romeo They won't know if it meets What the— if it's a solid creation or something that's been an application that's been written, if it does the function that they were aiming for, they'll count that as success without having any way to measure whether it's a complete insecure—
14:41Matt Coles it may leak data like crazy. It may, you know, leave open port on the internet. I mean, there's a host of things that, that can happen because while it does that core one, one core function, but it can't evaluate And I'm not talking, I guess I'm talking in general here, right? Chris Wright is a really important point. We've, this is sort of been a, you know, has been a topic that's come up fairly recently. Well, consistently now with AI is as AI takes on more responsibility, you have people who are not taking that responsibility and the set of what we don't know becomes larger over time.
15:18Izar Tarandach I think that, uh, one of the things that, uh, Chris just described is, I'm just checking here to make sure that I'm not crossing my streams. It's a thought experiment called the Chinese Room, right? Where you put somebody inside a, uh, a room and, uh, they have a huge book of instructions that's written in English, right? And people outside the room, they sleep. Chinese characters under the door, and the person inside doesn't understand Chinese, but they, they know how to look for instructions in the books that if you see squiggle A or squiggle B, then write it down and pass squiggle C back, right? And to the people outside, the person in the room appears to speak flawless Chinese, but the person in the room doesn't know Chinese. They just have this algorithm that translates things that come from the outside into some response that's Chinese.
16:16Matt Coles Yeah, they're interpreting, they're following instructions to interpret characters, not actually learning Chinese.
16:23Izar Tarandach So this was, uh, this was, uh, proposed by John Searle. Searle. I hope I'm saying the name right. 1980. And the core argument here is syntax versus semantics. And, uh, the conclusion that he got to was Programming is not thinking because you are basically just translating symbols, right?
16:48Matt Coles But you're, well, okay, I'll argue a little bit that in order to translate that symbols, you have to know what the effect of those symbols are. So you're translating an idea and a concept of I need to do something, I need to manage data and have an interact with the database and all that. And I program it.
17:08Izar Tarandach When it says programming is not thinking, it means the programming, the program itself, not the act of programming. The act of programming is definitely thinking. Sorry, my mistake.
17:17Matt Coles Sure. Sure. Yeah. And the, and the program itself doesn't, I mean, it's, it's just hardware executing bits, right? It doesn't understand what those bits are. It doesn't care what those bits are.
17:27Izar Tarandach Yep.
17:28Chris Romeo Right.
17:29Matt Coles It's the programmer who gives it meaning and it's the user who derives value and meaning from it.
17:36Izar Tarandach So one of the counterarguments to it is the systems reply that says that the man, the book, the room, and the paper, and eventually who wrote the book, they were thinking. So the book is condensed thinking of somebody else. Therefore, it is thinking. I don't know if I go there, but, uh, is it Turing compatible? I, I, I found it interesting how, how, uh, uh, basically, uh, a Chris brought up the same question and the same problem up to something that people have discussing like between weak AI and strong AI for decades already. Oh my God.
18:13Chris Romeo And I am a bit of a philosopher, so. Yep.
18:16Izar Tarandach We are all warrior from the philosophers for changing values of war and philosophy.
18:22Chris Romeo Okay.
18:24Matt Coles Sun Tzu in the, Sun Tzu in the Age of AI.
18:28Chris Romeo The art of cybersecurity.
18:30Izar Tarandach There is something there. Okay. So Chris, what's your episode?
18:36Chris Romeo It's going to be a shocker. It's going to be a surprise. It was, I think, our most controversial episode. No, it wasn't.
18:43Matt Coles Jazz is dead.
18:44Izar Tarandach No, it wasn't.
18:45Chris Romeo Jazz is dead. Yes. But I thought I'd revisit this in the context of, I know we're trying to stay away from AI today, but—
18:54Matt Coles Which episode is it? Just for our listeners.
18:56Chris Romeo I forgot. I was looking at something something else. Yeah, um, wait, I'll tell you in a second if you have a few minutes. Just, you know, hang around and we'll find it.
19:03Matt Coles You keep talking.
19:04Chris Romeo Well, now it's right here. I'm looking at it. It is— doesn't have a tag.
19:11Matt Coles Okay, so it's pre-May 4th.
19:14Chris Romeo It should have a tag. May 4th, uh, 2023, when we posted this one. It's called Reasonable Software Security: Do We Really Need DAST? And that was our whole DAST is dead, we don't need DAST, it's basically useless, and it's technology that doesn't really add a lot to the universe.
19:31Matt Coles Episode 13.
19:33Chris Romeo Episode 13. Okay. Thank you for that counter, Matt. Is, um, my LLM.
19:37Izar Tarandach When we were just learning.
19:40Chris Romeo We were, we were just learning. So, but I think this is even more true now, and I think it's expanded beyond DAST.
19:52Matt Coles DAST is deader.
19:54Chris Romeo I think other things are, I think other things are gonna be deader.
19:58Izar Tarandach Is it a parrot?
20:00Chris Romeo As well.
20:01Matt Coles There's a Monty Python episode in here somewhere. Yeah.
20:04Izar Tarandach Somewhere.
20:05Chris Romeo We are the knights that say nothing.
20:09Izar Tarandach Nyeh. We are the knights who say meh.
20:12Matt Coles Meh.
20:13Chris Romeo Meh.
20:14Matt Coles This product was alive.
20:20Chris Romeo Yes. But when we look across the AppSec tool space right now, We're still in the beginnings of AI integration and AI, you know, agentic AI inside of security tools is in various different states. Like nobody's really, we wouldn't say anybody's kind of figured it out and perfected it yet, but where does, I guess the bigger question is where does agentic AI in mixed with AppSec tools get us? Do we get to a point where human beings don't have to do false positive triage? Like, will we get there?
21:06Matt Coles Well, actually, you're, you're asking a more, there's a more fundamental question to ask. Does the tool, do the tools create systems that don't require verification?
21:15Chris Romeo Yeah. I mean, I'm just, I'm kind of piggybacking on our previous conversation. I'm drawing a connection.
21:24Matt Coles Uh, well, so you're asking, uh, do humans have to do fault— do humans have to triage results from tools and can they know what's a false positive? Yeah.
21:34Chris Romeo And will AI change the game of what's been the stressors or the things that developers have hated about AppSec tools historically, which is it gives me 10,000 findings. They don't have anything to do with what I'm doing. Is AI the answer? To that?
21:51Matt Coles I mean, AI is a first-pass filter, right? If you, well, okay, if you could pass, if you could take those results and say, here, intern, who's like, you know, first year in college learning about computer science, here, tell me what's real, right? You would use it as a learning opportunity. And that person, by the way, could be stellar and know that, right? That would be great. but the AI is in that same boat, right? How does it know? It can, there's some things that can know very well, right? Like, does that service have an external port assigned or an external interface assigned? That's easy to know.
22:32Chris Romeo Right.
22:33Matt Coles But, but certain things require context. Like this service is running as root. Does it need to be right? That's a, that's not a. That's not a rule-based answer.
22:47Chris Romeo But won't it, won't it be able to go and absorb all the code first?
22:50Izar Tarandach Wait, wait, wait, wait.
22:51Chris Romeo The abstract tool of the future, wouldn't you go have it absorb the code base and then run test cases against the app running application, knowing what the code is doing behind the scenes?
23:01Matt Coles Sure. But do you even have to do verification at that point? I mean, and why do you have to do DAST? Like, why do you have to do dynamic application testing? I'm expanding.
23:10Chris Romeo Yeah, I'm expanding. I'm call, I'm going more generic with abstract tools. Okay.
23:13Izar Tarandach So, Sure. So let's look at it again, uh, at the way tools work and the way the quote unquote intelligence works. And then let's bring it to the intern again. So the way that things have been going so far in the universe, people found problems. For every problem, a pattern emerges. For every pattern, a rule. Is specified. Those rules are added to engines that run with frameworks. Those frameworks give the engine the access to the data that it needs to run the rule. If the rule fires, then woo, positive. If the rule doesn't fire, woo, negative. And it has been like that for almost 30 years now, right? And the one tool that we declared dead. Is the one that goes and pokes at things from the outside. Now, past few months we have seen, I, I at least have seen people singing the praises of, uh, what's it called? AI-fueled, AI-fueled, uh, uh, DAST, or even people telling their agents to create a very customized best, uh, harness to test their work.
24:37Matt Coles Okay.
24:38Izar Tarandach And some people are happy, some people aren't, right? But I, I, I think that what that, uh, ignores, and here's where the intern comes. If we go Matt's way and bring in an intern to look at the results of these tools and decide what's good, what's not. Before I give the, the intern access to the tool and, uh, show how to use it, I'm going to have a very, very interesting sit down with them and explain that, listen, it's okay to make mistakes. It's fine to make mistakes. But the thing is, we really want to test this thing that we have before we put it out there, because otherwise there's going to be a lot of risks, a lot of impacts. We don't want that. So do me a favor. When in doubt, raise the flag. And there will be a bunch of different processes running in that intern. Fear, commitment, uh, the idea of doing a good, good job, wanting to learn that I can trust the fact that if something smells bad, the intern will come and say, hey, I need somebody with more, uh, more experience to take a look at this and tell me what's happening here. Explain to me, teach me, and next time I'm going to learn. The LLM will simply decide what it decides.
25:59Chris Romeo So are you calling for an AI-driven AppSec testing tool which will do its best, but then know to kick things up to a human? If it doesn't—
26:14Matt Coles I mean, a human in the loop.
26:15Izar Tarandach That would be the human in the loop, yeah.
26:17Chris Romeo Human in the loop. but it'll address some things that it clearly says, yes, I understand these things. I get them. And others, it will bounce them to the human.
26:27Izar Tarandach I think that LLMs nowadays bring me back to me at 15, 16, where I was consistently authoritatively wrong.
26:38Matt Coles Oh, I know this.
26:41Chris Romeo Right?
26:42Matt Coles Like, like, hey, I know Unix.
26:46Chris Romeo We have to get to the age that we all sit at now to admit it.
26:49Izar Tarandach Yeah. And, and my point is, you know, I prefer somebody telling me I don't know than somebody telling me, of course I know it and it's okay. And it's not.
27:00Chris Romeo And you're saying that's what the LLM will do today is be like, of course I'm saying that's what the LLM might do.
27:07Izar Tarandach And that to me, that's the other side of non-determinism that makes the whole thing questionable.
27:16Matt Coles Well, and the, the fact that the AI, for some reason, and by the way, I'm, I'm curi— I'm curious and I wonder if anybody who watches this, these episode or listens to these episodes, uh, happens to be an expert in the construction of AI systems. But what in the code of AI makes AI want to please? It wants to say yes.
27:40Chris Romeo Right?
27:41Matt Coles It's confidently correct about everything and upbeat, right? The conf— building— one of the other, one of that other topic on that Register article was the AI builds confidence. It wasn't the Register, sorry, it was another article. But the AI's first job is to build confidence by, you know, in you that the results will be correct. And then it's confidently correct about the, or confident about the things that it that it outputs as correct.
28:11Izar Tarandach So I, I read—
28:11Matt Coles and what in the code actually makes that possible? Because machine learning doesn't do that. Machine learning is just a value prop. It says, I think this is correct. In fact, it doesn't say I think. It says this matches my criteria, you know, 78% of the time, or it has a match confidence. And somewhere it got translated to yes, This is great. And it's that lack of the ability to be negative and self-reflecting and to be critical to say, this may not be right. I need to ask a human. I need to ask a more senior individual for assistance.
29:00Izar Tarandach So Apparently the, that there's a name to that and it's the psychophantic effect and it comes from training and in part it's a human bias because there's an approval trap. So people who are training these models, they have a psychological bias towards validation. We are more likely to give a high rating to an answer that agrees with what we think than not. And the models pick that up. Then there is prioritizing helpfulness over honesty. I'm reading off my LLM here where I asked, why are they sycophantic? And model developers typically optimize for 3 pillars, helpfulness, honesty, and harmlessness. I didn't even know that that was a word. And then there's a—
29:48Chris Romeo Harmlessness.
29:49Matt Coles Harmlessness. Do no harm. Yep.
29:52Izar Tarandach And then there is a deference to the person who's asking. right? So LLMs consider the, the frameworks call the LLMs themselves, sorry, they consider people to be peers or assistants or people to be assisted. And then there is a psychophancy as a form of reward hacking. The model finds a shortcut to a high score instead of doing the hard work of verifying facts or arguing a complex point. It can simply praise the user's input. Right. So it tends to go and answer the way that you want it to be answering.
30:30Matt Coles So that you click the thumbs up versus thumbs down.
30:36Izar Tarandach Yep. Because it wants to be rewarded for probably doing some good work.
30:42Chris Romeo Yeah.
30:42Matt Coles But all of that is injected behavior because the LLM doesn't have It's trained behavior. It's trained, but it's, there's code underneath and somebody wrote that code.
30:55Izar Tarandach There is weights underneath. So the dataset that was used to train it already has some of that bias, already has some of those psychophantic characteristics that influences the weights that are present in the model. And that influences the way that the model finds the shortest path or the shortest distance between where you are and where you want to be.
31:14Matt Coles But you're implying—
31:15Izar Tarandach and that's all I understand from the math. Please don't kill me.
31:18Matt Coles You're implying that the transformer system that powers generative AI has a sense of reward.
31:29Izar Tarandach No, but the, from my understanding, again, I'm sorry, I could be totally wrong here, but my understanding is that higher scores are rewarded. Are rewarded. And those usually mean that somebody is agreeing with what you just said because they are closely, that they are more, that there is less distance between you and the next predictable thing in the latent space.
32:00Matt Coles But that reward is simply the mat— the choice matches a prediction. It's a math problem.
32:09Izar Tarandach Yes.
32:09Matt Coles It's a mathematical problem. Not a feeling problem. Not a feeling problem.
32:12Izar Tarandach There's no feeling there. The only feeling here is being frustrated by the answer.
32:16Matt Coles And so, and so now back to, back to Chris's question, right? So DAST requires an understanding of a system, the way it works, but also the way it's supposed to work, right? What you expect it to do. And the AI, unless it has that knowledge, will tell you it's doing what it's expected to do because it doesn't— either doesn't know or doesn't have the ability to know unless you have that human in the loop. But can the human in the loop, going back to the episode that I brought up, at some point will the human know what the tool is saying, right, about the results? So the DAS tool You need the, the problem with DAST was the, was the poor detection, right? And the generation of false positives. Well, if you look at the false positives, what were those false positives? Was that a rule problem or was that an interpretation problem?
33:18Izar Tarandach Could be both. Could be.
33:20Matt Coles And it could be both. And which one is worse?
33:24Izar Tarandach The one that hides the most impactful issues. Or the one you spend the most time on, or the one that you spend the most time on. Yes.
33:32Matt Coles Right.
33:32Izar Tarandach Right.
33:32Matt Coles And so is that, is that an interpretation problem or is that a rule problem? If the rules are bad, you fix the rule. You don't need interpretation, but if, if it's fuzzy, you can't fix the rule and it becomes an entirely an interpretation problem, like the least privilege. I can know that this is running as root. Is that the least privilege that it needs?
33:55Izar Tarandach But look, that, that has to be a rule problem because if we go back in time to the, the time when Das ruled the earth, and there was such a time, if you look at all different, uh, uh, products, the only difference between them were, were the rules because they all could do the same thing. They all could open a page. They all could parse HTML. They all could blah. They all could blah.
34:16Chris Romeo Mm-hmm.
34:17Izar Tarandach The only difference was the rules, which rules they embedded. Yep, exactly. The, the, my, my question now is by putting AI in the loop, are we creating more rules? Are we interpreting rules better? Are we instrumenting the interpretation of the rules? What, what is it that AI is adding to things now that some people think is good, some people think it's bad? That's the good thing about AI. We, it, it doesn't look like anything is absolute. For everyone saying, hey, this is great, there'll be people saying, hey, this is not great.
34:53Matt Coles Which is really ironic because AI is entirely built on probability. Nothing is absolute.
34:58Chris Romeo No, it is not great.
35:00Izar Tarandach It's a translational thing, right? We take the probability coming out of the AI and we apply it to the results that we get. But there is, I'm not going to say the name of the outfit, but there is a very respectable and respected outfit out there that among other things does a lot of pentesting. Uh, and if you just Google for it, you're going to find that they have created a whole framework for their use of agents and published on GitHub. And they have some very good stuff regarding DAST. And to be sure, I'm certainly not ashamed in any way, shape, or form to say, if they trust it, I trust them, right? So there must be something good coming out of this. Is it enough to resurrect DAST? I don't know, but they seem to be very happy.
35:58Matt Coles Is it the value? The, I guess it goes back to, are they improving detection capability, meaning the core rules underneath, or are they improving detection capability and reducing false positives because the AI can do can do the interpretation to a degree that removes the human from it? And what's the impact of doing that?
36:17Izar Tarandach Look, I, I, I can't attest to that, but I can translate the, the, the problem to something that you and I know very well. I recently published, uh, uh, threat modeling skills for agents based on PyTM. PyTM is a rule based engine, right? So you, you, you put your system in, you have rules, the rules get evaluated, you get answers, completely deterministic. You put the same stuff in, you will always get the same stuff out.
36:50Matt Coles You put your left foot in, you get your left foot out.
36:52Izar Tarandach Exactly.
36:54Chris Romeo And talking about the hokey pokey.
36:56Matt Coles Yeah. PyTM for pokey pokey.
36:58Izar Tarandach The PyTM pokey pokey. And, um, I, I wrapped that. On a skill that will basically reduce, much like what we, we, we describing here, it will reduce the amount of, uh, uh, cognitive jumps between having a system and describing it in a certain syntax, the way PyTM works. But not only that, it's going to ask you questions. It's not just going to say, give me your description. It's going to actually lead you down the, the, the rabbit hole and ask you about the system until it's satisfied that it has a good description, and then it will work with PyTM to get you the results. Why did I choose to do it that way? Because to me, the most important bit was people don't know what they have. And if I have somebody who can ask them, much as we do when we lead live threat modeling sessions, right? The biggest part is not to find out what could go wrong. It is what are we building?
37:58Matt Coles Right. 'Cause without, so it's a, it's a Geico problem, right?
38:02Izar Tarandach Exactly. It's the Geico problem.
38:03Chris Romeo Car insurance. Yeah. Sorry, I couldn't help it. I can't help it.
38:09Matt Coles Geico. Garbage in, it's garbage out.
38:11Izar Tarandach It's little, little lizards running around your code. It's not the garbage in, garbage out.
38:16Matt Coles Yeah. If you have a bad model or you have an incomplete model, you get garbage or incompleteness on the outside.
38:22Chris Romeo Exactly. The beautiful thing is it'll think it's beautiful. It will, it will sending you, it will, nobody's baby is ever, ever ugly.
38:30Matt Coles Right.
38:30Izar Tarandach So, or I think that the point that I'm trying to make is that because of that randomness, I think, or because of the uncertainty, let's put it like that in the interpretation of results in something that is so mired in uncertainty from the beginning, like threat modeling. Just moving a second away from, from that. That's why I think that the approach that, uh, Davichi had, where the AI was basically helping you along creating the threat model or helping you along interpreting the results or helping you along creating rules, but it wasn't describing the system. It wasn't running the, the, the engine. It wasn't deciding things for you. The approach that I took with the skills for, uh, for, uh, PyTM. Where the, the, the, the AI is basically reducing the cognitive load that you already have in building these models and in presenting the results rather than actually doing the threat model, because that's something that we, we, we can't get away from. It's very rare the occasion where we find something new at the, the, the low-hanging fruit of threat modeling. Business level, we find it all the time. But what other people would call the specs, it's very, come on, if we are honest with ourselves, you can get to a certain amount of things that you have, this is the table stakes, you have to have these things to have a secure system. And if you have those things, then your threat model, many times at some levels will look at you and say, yeah, I haven't found anything, you covered all the bases. Right?
40:12Chris Romeo I still haven't found what I'm looking for.
40:15Izar Tarandach Exactly. And in his case—
40:17Matt Coles Security has a soundtrack.
40:20Izar Tarandach It's threat model prioritization and the car keys. Those are the two things that keep eluding Bono repeatedly.
40:29Matt Coles That's because he wears his sunglasses all the time. I mean, come on.
40:34Izar Tarandach I wear my sunglasses at night.
40:38Matt Coles You have to turn down the inside lamps at night because your transition lenses are so dark.
40:44Chris Romeo Okay, come on, old people, your transition lenses. Come on, get out of here. You're buying your trifocals.
40:50Izar Tarandach Do I get to very quick go to my episode?
40:52Chris Romeo Yeah, yeah, let's do it. What is it you wanted to hit?
40:54Izar Tarandach So I am going to go back to 2024, the dark ages of the Security Table, where we had the AppSec Resolutions episode, where we made predictions. I wore my Nostradamus garb. And we made predictions and apparently there is a status to those predictions. So 2024, we predicted that a major security breach or leak of a popular AI involving the disclosure of unknown privacy details. The status, true. There's a concern about AI slurping that we spoke about at 8 minutes and 13 seconds that has become a central theme in global AI regulation discussion.
41:35Matt Coles Episode 39. We're ahead of the game.
41:39Izar Tarandach Dan, we predicted—
41:41Chris Romeo With regulation, we were number one.
41:44Izar Tarandach We predicted that S-bonds would reach its peak of acceptability and move from hype to a more realistic and practical application. Status: true. The prediction that S-bonds—
41:58Chris Romeo The crowd goes wild.
41:59Izar Tarandach The prediction that S-bonds might become a frustrated swear word in offices due to the legal and management overhead proved accurate. Curate.
42:07Chris Romeo Which was that? What was that at the beginning?
42:10Izar Tarandach The prediction that SBOM might become a first traded swear word.
42:14Matt Coles It's a four-letter word, yes.
42:16Izar Tarandach Then we said that a resolution was made to push for secure coding education. No, no, we said there's going to be secure coding in education. The status, partially true slash ongoing. While a national curriculum hasn't replaced CS101, There has been a significant increase in industry-academic partnerships. OpenSSF and more vocational schools. Then we said that, and you're gonna love this one, the new acronym for DAST. Okay. We said that, uh, uh, what's it? Someone would invent a new acronym for DAST to revitalize the old technology. Status mixed. While the term DAST still exists, the market has largely pivoted towards ASPM. And AI-native testing tools that attempt to fulfill the no noise requirement.
43:07Matt Coles Wait, ASPM is not AI though, right? I mean, ASPM is just application security posture management. It's different from DevTest in such a way.
43:15Chris Romeo Hey, let us book the win, Matt.
43:18Matt Coles Wait, you're not the pessimist? What kind of security person are you? Wait, wait.
43:23Chris Romeo I like winning.
43:23Izar Tarandach The last one, the one that says that we keep writing the frontier of this whole thing. The quantum LLM joke. The host joked about quantum LLMs being the next buzzword for 2025. Status: on the horizon. Not yet a mainstream commercial reality, quantum machine learning has indeed become a major research focus in late 2025 and early 2026, often appearing in the future tech keynotes the hosts were mocking.
43:56Chris Romeo We mocked keynotes? I don't think that would happen.
44:00Matt Coles Yeah, we did that.
44:01Chris Romeo Yeah, we did.
44:02Izar Tarandach I have never mocked a keynote in my life this week.
44:06Chris Romeo This week? You sure? This week?
44:09Matt Coles Wait, you're behind the curve already?
44:12Izar Tarandach Guys, it's Monday. It's Monday. Give me a chance.
44:16Matt Coles Are you sure?
44:17Chris Romeo What are you trying to say? You don't mock over the weekend?
44:20Matt Coles Are you mocking me?
44:23Izar Tarandach I only mock professionally.
44:25Matt Coles If you give a key— if you give a keynote, you know, you are fair game for, for mocking.
44:29Chris Romeo True, true, true. Well, I'm glad to hear that our AppSec resolutions were 100%, 100% spot on. We apparently can see the future in ways that other people can't. So you should really listen to this podcast just for the future telling. That's happening. And what's that site where you can like place a bet on anything?
44:53Matt Coles Uh, FanDuel?
44:55Chris Romeo No, FanDuel's from sports.
44:57Izar Tarandach The what though?
44:58Chris Romeo Thanks for the free plug. What is the one where you can do, there's a place where you can do a, you can place a wager on anything.
45:04Izar Tarandach Oh, oh yeah. Kashi.
45:06Chris Romeo No, that's not it. It's a big company now.
45:10Izar Tarandach Yeah. No, no, it's called Kashi. I, I had it appear on the Lion Kingdom sometimes.
45:14Chris Romeo There's another one.
45:15Matt Coles Don't they make energy bars? Is Kashi is a, an energy bar company.
45:19Chris Romeo That sounds incredible.
45:20Izar Tarandach No, that's cash.
45:22Matt Coles No, that's a nut. You're a nut.
45:26Chris Romeo You're a nut. I'm checking with my AI here.
45:33Matt Coles Oh, come on.
45:34Chris Romeo Oh, Kashi, Kashi is Polymarket is what I was thinking.
45:37Matt Coles Polymarket.
45:38Chris Romeo Oh yeah. Yeah. Yeah.
45:39Izar Tarandach Yeah.
45:39Chris Romeo We should go make some wagers on our 2026 resolutions so that we can make some cash.
45:47Matt Coles You know, we should also, so in season 3, episode 4, back in March of 2025, we talked about vibe coding and what can go wrong in vibe coding. We should see what went wrong with vibe coding.
45:59Chris Romeo So you're trying to say we were ahead of the game.
46:01Matt Coles I'm saying we were, we were very predictive.
46:03Izar Tarandach Yes, absolutely. No, that, that, that's kicking the puppy. Come on. All right.
46:08Chris Romeo Do not kick a puppy. All right. We have rules here at the Security Table. At the original Security Daily.
46:14Matt Coles At the original Security Daily.
46:15Chris Romeo We don't kick puppies. I'll just let, I'll just let you think about no, the alternative to that.
46:22Matt Coles We should look, we should look over the past 100 episodes and look at all the movie references that were made and put together our, our movie night lists.
46:30Chris Romeo Yeah, that's a good idea.
46:31Izar Tarandach We get some good ones.
46:32Matt Coles We get, we have a song playlist.
46:35Chris Romeo That's your homework. That's your homework.
46:36Matt Coles I'll, I'll just ask the AI to do that.
46:39Izar Tarandach If you haven't seen Dark City yet, it's going away from the streamer that it actually is right now.
46:45Chris Romeo And it's free.
46:47Izar Tarandach Dark City? So, Dark City.
46:48Chris Romeo I thought you said dark and seedy, and I was like, ooh, what do you want to see?
46:51Izar Tarandach No, no, no, no, no, no, Dark City.
46:53Matt Coles Dark City.
46:54Chris Romeo Dark City. That sounds familiar. That's the movie reference of the week from the Security Table. Well, folks, thanks for 100 episodes. Thanks for being there and tuning in and listening. You make it the reason why we do the things we do. If not, we would just be 3 weird—
47:12Izar Tarandach at least the guy in Australia.
47:13Matt Coles Yeah.
47:14Izar Tarandach And the one in Canada.
47:15Matt Coles Yeah.
47:16Chris Romeo Yeah. We'd just be talking to ourselves on a Zoom call and nobody would ever know what happened, but these very similar conversations would be occurring.
47:25Izar Tarandach So, and hey, for the Next 100, if anybody wants to think about sponsoring us, please feel free.
47:31Chris Romeo Trying to find product placement, get a product placement there.
47:40Izar Tarandach So if only somebody here worked at a major broadcasting corporation with a huge podcast catalog.
47:47Matt Coles You know, I was just noticing that we're all— I don't know why I just noticed this, but we're all using 3 different technologies for our audio today.
47:56Izar Tarandach We are?
47:57Matt Coles Yes. You have bone conduction. He has in-ear buds and I have a traditional headset wired with a mic.
48:06Chris Romeo So that's just 3 different— I'm, I'm gonna draw some conclusions here. So like, Izar's the cutting edge, I'm the hipster, and you're the old guy.
48:15Matt Coles Which is ironic cuz I'm the young guy.
48:17Izar Tarandach Which is ironic because it's actually the other way around.
48:21Chris Romeo There's, there's the name of the episode.
48:24Izar Tarandach I, I, I'm the old guy trying to be the hipster who's in the cutting edge of the technology.
48:30Chris Romeo That's a good, good name for this episode though. The, the technology, the hipster, and the old guy. That's got a good rhythm.
48:39Izar Tarandach The ugly, the bad, the ugly, and the— what was the other one?
48:42Chris Romeo The good, good, the bad, and the ugly.
48:43Izar Tarandach The good, the bad, and the ugly.
48:44Chris Romeo Yeah. Yep. Well folks, thanks for listening. Thanks for being a part of our audience. We appreciate it. Reach out to us on the socials or anything else. Heck, if you want, if you're listening to this and you wanna be a guest, you're on.
48:56Izar Tarandach No, totally. Just, we want to know about you.
49:00Chris Romeo We're thinking about having a call-in line. We're going to the caller, caller 100, you win a free Dash subscription for a year.
49:09Matt Coles Yeah, we'll be, we'll be on a 50— we'll be on a 15-second, uh, time delay for that.
49:13Chris Romeo Oh, we'd have to be because these are— it's kind of amazing.
49:18Izar Tarandach Hey, I'm delayed, what can I tell you?
49:22Chris Romeo Oh, that— folks, have a great Next 100. Hopefully you'll follow us along for the ride.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
