What If AI Never Happened? The AppSec Reality Check
With Chris Romeo, Izar Tarandach, and Matt Coles
In this episode, we explore a simple but surprisingly deep question: what would application security look like if generative AI never existed? We break down how AppSec might still rely on deterministic, rule-based approaches, what we might gain in structure and rigor, and what we’d lose in speed, scale, and accessibility.
About this episode
In this episode, we explore a simple but surprisingly deep question: what would application security look like if generative AI never existed? We break down how AppSec might still rely on deterministic, rule-based approaches, what we might gain in structure and rigor, and what we’d lose in speed, scale, and accessibility. Along the way, we debate whether AI is truly improving security or just accelerating existing problems, from “vibe coding” and false confidence in results to the growing gap between finding and fixing vulnerabilities.
We also get into the tension between human-driven security practices and AI-assisted workflows, and whether the biggest challenges in AppSec are actually technical at all or still rooted in people and process. Plus, things take a turn as we let AI weigh in…and roast us a bit in the process.
Per usual, it’s a mix of thoughtful discussion, strong opinions, and a little chaos.
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 47 min conversation
0:11Chris Romeo Hey folks, welcome to another episode of The Security Table. This is Chris Romeo, joined as always by Isar Gemini. No, no, no. Isar Claude and Matt Gemini. They, they have, uh, pitched their allegiance to a particular, uh, AI. And, uh, for purposes of this conversation, I will be AI-less because I will try to represent the human side of the world. Uh, the, the challenge Matt's throwing before us is to ask and ponder the question: what if generative AI did not exist? Now, specifically in the domain of AppSec and cybersecurity, what would our world look like right now if this generative AI thing hadn't started running off with all of the future?
1:07Izar Tarandach Oh God.
1:08Matt Coles If it hadn't changed the future, right? Timeline-wise, uh, we're gonna call this one the man in the high AI. If things around 2023 had kept on going as it was versus split off into the AI world. The AI universe?
1:22Chris Romeo It's a good question.
1:25Izar Tarandach It's, uh, yeah.
1:27Chris Romeo Would, is time travel required to answer this or no?
1:31Izar Tarandach I'm still trying to decide which one of them, which one of them is going to be the bizarre award, the one with or the one without.
1:38Matt Coles We could be predictive and generative.
1:40Chris Romeo I just like the idea of time travel and I never have a reason to do it. So, I mean, it's, you never had a reason to do time travel?
1:49Izar Tarandach No, I am too.
1:50Chris Romeo Everything.
1:50Matt Coles I mean, I mean, so we, we, if we're gonna get into that conversation, we need to figure out what ti— how you feel about timelines in time travel. Do you go back and can you change the past or can you never, yeah, I can never observe. You can only observe.
2:05Izar Tarandach You can't change.
2:05Chris Romeo Follow the Back to the Future rules.
2:10Izar Tarandach So this weekend I tried to have a, and I didn't try, I actually managed to, I had a conversation with my AI. For you to explain to me the difference between the multi-world theory and the multi-universe theory. And that was an interesting conversation to have.
2:25Matt Coles As opposed to say Star Trek, when they went back to the past, they didn't change the future, did they? Not in all cases.
2:33Izar Tarandach Because there was a branch in there.
2:34Matt Coles Right.
2:35Chris Romeo Yeah. Proving Star Trek is better. On the other hand, Back to the Future.
2:41Izar Tarandach Back to the Future does change the future back.
2:44Matt Coles It does.
2:44Izar Tarandach Because there's no branching.
2:46Chris Romeo Let's see what other, what other, uh, although interestingly, when Dr.
2:51Matt Coles Emmett went back to the 1800s, actually they had to fix, that's right. They had to fix the timeline in order to not break the future.
3:00Izar Tarandach That's the timeline. Yeah.
3:01Matt Coles Right. Yeah.
3:01Izar Tarandach Mm-hmm. All right. So back to our AI, right?
3:05Matt Coles So if, if AI never took us on the alternative path. What would, what would cybersecurity, what would AppSec look like today?
3:16Izar Tarandach Well, for, for one thing, we would be getting much less spam because less people would be trying to sell us tools that they just AI slopped the heck out of it.
3:31Matt Coles You mean, you mean you would, we'd be on the same path. We would have been on the same path of new tools coming out, trying to create new domains of security.
3:41Izar Tarandach Yeah, but slower.
3:42Matt Coles But slower.
3:44Izar Tarandach And we wouldn't have a complete domain that has flourished with people having 20 years of experience on a 5-year field. And, uh, we would be treading to a lot less material that we have to decide, is this something that I have to pay attention to, or less. Mm-hmm.
4:06Chris Romeo So you're saying there's been a glut of products and companies that wouldn't exist without AI because AI is being used to create them and to justify them. What do you mean by that?
4:21Matt Coles You need, you need this because the AI explosion, and so you need this, right?
4:27Izar Tarandach There is a whole lot of stuff that we have wanted for a long time. That has been instantia— instantiated by the promise of AI to be able to gather a lot of contextual data and make it reasonably easy to understand and process. All those wouldn't be around here if we didn't have the capabilities that we have today. Then there is all the classification stuff. Promising to put things into bins in ways that the regex couldn't, which also wouldn't be around.
5:07Matt Coles Well, I mean, all right, so alternatively, machine learning, right, which is not GenAI— GenAI is an application of machine learning— machine learning could have advanced still, right? That's the classification problem. Could have solved the classification problem. And it did, but I think that it would've been a lot less accessible, right?
5:29Izar Tarandach Exactly. The way that it came out of labs came because of that specific application of it that made it all of a sudden much more accessible. If you went back in time, uh, a couple of years and asked people what's PyTorch, they would be looking at you like, I don't know, something that, uh, Elon wrote to write a flamethrower. I don't know.
5:49Matt Coles Right.
5:51Izar Tarandach So, That there is that too. But I like the answers that we got from our AIs. Let's not say which AIs we're using and people can guess in the end from the answers.
6:05Chris Romeo What do they win?
6:08Izar Tarandach The satisfaction of being right or wrong.
6:11Chris Romeo So nothing.
6:12Izar Tarandach Okay.
6:12Chris Romeo Nothing.
6:13Matt Coles Nothing. Nothing. There's, it's free. It's free to enter and there's no prize.
6:17Izar Tarandach Exactly. Exactly. Rules apply.
6:22Matt Coles Well, you can be assigned points, but the points don't, you can be assigned points, but the points don't matter.
6:27Izar Tarandach Exactly. Exactly.
6:29Chris Romeo Fair.
6:30Izar Tarandach Whose AI is it anyway?
6:34Chris Romeo Okay.
6:34Izar Tarandach So we asked it, what would it be? And Matt, what did you get?
6:40Matt Coles Ah, well, uh, I got basically it would be manual. Reactive and bottlenecked by humans.
6:53Chris Romeo Oh, how dare us humans to bottleneck something.
6:58Matt Coles It would be an era of deterministic security as a bad— and then, and that's, and that, and the way that that's worded is as a bad thing. And if you think about that very carefully, deterministic security as a negative, versus the alternative of non-deterministic security as a positive. Think about that for a moment.
7:26Izar Tarandach That's interesting.
7:26Chris Romeo That, yeah, hold on. That's actually a positive. Cause when I think deterministic, I think predictable. Yes. Yes. That's the synonym I like to use for that just to make it make sense in my head.
7:39Izar Tarandach Yes.
7:39Matt Coles You run it, you run the same function multiple times and you get the same output.
7:43Chris Romeo Right. Yeah. That sounds like a good thing.
7:46Izar Tarandach It is. Not a bad thing.
7:48Chris Romeo Yeah.
7:48Izar Tarandach But consider this, who's answering and what their point of view is. You're asking something to tell you how the world would be if it didn't exist itself. So you would expect that it's going to say, well, it's going to be a bad thing because I'm not around. And one way to say I'm bad, bad thing, I'm not around if I am AI is, hold on, we're not talking, we're not talking AGI, we're talking generative AI. This is not AGI. This, this, this is just implicit bias of the thing.
8:15Matt Coles Why would an algorithm—
8:17Izar Tarandach Because it wants to sound helpful, right? So for it being helpful is it being helpful, something existing that is helpful. If that something is not there, then—
8:28Matt Coles Probably we need to change the query then to say, if generative AI wasn't at use in AppSec, it still existed, but it was not available for AppSec.
8:41Izar Tarandach So I got amused because I gave exactly word by word the same prompt to mine. And one of the things that it says is that the foundational tooling of application security, blah, blah, blah, blah, blah, existed before, blah, blah, blah, blah, blah. None of these are GenAI techniques. They are deterministic or heuristic-driven analysis engines. Then it says, they would exist exactly as they do and arguably with more investment and refinement if the industry hadn't been distracted by the shiny object.
9:13Matt Coles So yours had no problem being existential.
9:16Chris Romeo Nope.
9:16Izar Tarandach Mine has absolutely no problem being self-reflective. And I think this thing is learning from me.
9:23Chris Romeo I'm getting seriously, uh, so what, it's cynical?
9:29Izar Tarandach You tell me from the next paragraph. Threat modeling, your domain specifically, is a structured reasoning exercise. PyTM doesn't use GenAI. Stride doesn't. The Threat Modeling Manifesto doesn't. The discipline would be exactly where it is because it is— its progress has been driven by methodology and practitioner adoption, not by token prediction. Yep.
9:55Chris Romeo I don't know that I agree with that statement though, because you've at least seen in the threat modeling space generative AI be applied to various tools that help make threat modeling work, like DaVinci, to try to make the grunt work side of threat modeling easier. So it's, it's an, I, I think there has been some positive steps forward and I don't think it would truly be practitioner kind of driven.
10:32Izar Tarandach And I agree with you, but as you bring DaVinci up, It was AI infused, so it was around the process, not around the threat modeling itself. So it made the process easier, but not the threat modeling more exact or more perfect or something else.
10:48Matt Coles Is that the, is that the term we're using now is AI infused?
10:51Chris Romeo Oh, I came up with this. It is.
10:53Izar Tarandach Yeah.
10:53Matt Coles Okay.
10:53Izar Tarandach Yeah, exactly. Actually, I think I heard Chris Romey original.
10:57Chris Romeo I think I heard someone else say it and I repeated it twice, 3 times as required and then made it my own.
11:02Matt Coles As opposed to AI assisted. Anyhow, I updated my, I updated my query and, uh, I basically got the same, same results as before, but then, but then an updated verdict. Oh, in a world, in a world, in a world, yeah, without, without GenAI, AppSec would be more principled but less productive. We would see a shift towards formal verification and strong typing as the only way to mathemati— mathematically prove security rather than the best effort probabilistic security we often see today.
11:42Chris Romeo Wrong.
11:44Matt Coles Well, formal verification and strong typing would be mathematical ways of proving security and it would be structured and it would be principled.
11:53Chris Romeo And they're so highly inefficient that you'll never release an application as opposed to using the probabilistic techniques that GenAI offer. Boy, I think I'm back on GenAI's team at this point. If formal verification is what we'd be stuck with without it, which is not the case because we've had that, it's been available to us since the '60s, right? This idea of, it's just that technology or applications became so complicated that to formally verify even an app that's running on one of our phones right now, would take 6 months.
12:32Matt Coles Well, alternatively, you'd have to design things in a more structured way. And, you know, the benefit of Agile to be able to release quickly, but the downside of Agile to be able to release quickly. We've had this conversation before multiple times. Right.
12:49Chris Romeo And in our sleep also.
12:51Matt Coles Oh no, Agile.
12:53Chris Romeo Yeah.
12:54Izar Tarandach So apparently my AI is a bit more refined than Matt's, for those who are following the plot. And I am having—
13:03Matt Coles you're using like the fancy, a fancy version.
13:07Chris Romeo Yeah.
13:07Izar Tarandach All right. And, and it's, it's tuned to, to an extent. And apparently it's been fed with the shit that I've been writing lately. So my prompt at the end of the answer, it gives a synthesis of the whole thing that it gave before.
13:22Chris Romeo Aye, aye, aye, aye, aye.
13:24Izar Tarandach The domain of application security would look substantially similar to what it looks like today. Because its core problems are structural, methodological, and cultural, not generative. The areas where GenAI has provided real value are primarily in acceleration—faster triage, faster drafting, faster pattern recognition—rather than in capability. The trade-off is that some of that acceleration has come with new risk surfaces and the dilution of rigor.
13:56Chris Romeo The delusion of rigor.
13:58Matt Coles So wait, delusion or disillusion? Delusion. Delusion.
14:01Izar Tarandach Delusion. Okay. The challenges that would remain hardest without GenAI are the ones that are hardest with it. Scaling security expertise across large development organizations, making threat modeling a consistent practice rather than a checkbox, and closing the gap between we found the vulnerability and we fixed the vulnerability. Those are people and process problems. They were, they were there before LLMs and they still are. Hmm. Welcome to my AI TED Talk. It could have been mine.
14:36Chris Romeo Now I, I do fear that your AI has been, that you're leading the witness here. Compromised. Because it seems like, it seems to be answering in a way that I would expect you to answer.
14:53Izar Tarandach Yeah. Am I an AI?
14:55Chris Romeo Maybe that's what we need to test. Uh, how do we, Matt, how do we reproduce the Turing test against these?
15:05Matt Coles I, I don't know. Go, you ask, ask AI, I guess.
15:09Chris Romeo We have to, well, that, that's just compromising this whole experiment, Matt. Um, let's see.
15:14Izar Tarandach Wait, isn't that the premise of, uh, of, uh, Blade Runner? The reverse Turing test?
15:21Chris Romeo Or is it the direct Turing test? I never thought of it as the reverse Turing test. Is it? I don't know.
15:30Izar Tarandach It's, yeah, 'cause you're trying to, you're trying to prove that you're not a, a, a replicant. Yeah. So it's the reverse.
15:36Matt Coles There is a, there is a, there's a site. I would not, I don't know if I would go to it, but Turing, TuringTest.live. Play the classic Turing test in a modern format featuring conversations with advanced LLMs like GPT-4 and Claude.
15:54Chris Romeo Wait, GPT-4 is not advanced LLM. Get outta here. False advertising. Don't talk too loud.
16:04Izar Tarandach Okay, so where do we land with the, the answers?
16:08Matt Coles So, The common theme, I think, from mine is predictive and probabilistic today versus deterministic in the alternative world and rule-based, right? So everything being rule-based and obviously heavy human in the loop. So is that a good thing or a bad thing? I think we know that deterministic output is, is a positive in many cases, right? Especially with threat modeling and having humans in the loop. Obviously that's our need to have control. Right. But we can probably ignore that one, right? If we fully gave into the AI and let it do its thing, I mean everything from writing software to fixing software to fielding vulnerabilities and handling incidents, stuff would move along. Would it be perfect? No.
17:22Chris Romeo How long before the world would end?
17:24Izar Tarandach We'd be riding horses and how long before instead of MoatBook, there would be, What was the name of the other site? The one for programmers? Stack Overflow. Stack Overflow for agents. So it's funny you mentioned that.
17:43Matt Coles So one of the output here was the Stack Overflow remediation loop.
17:49Izar Tarandach Get out of my head, AI.
17:51Matt Coles Without GenAI-powered coding assistance to suggest secure patches. I don't know if it does that, but Sure, the remediation cycle would be significantly slower. So when a developer is told they have a cross-site scripting vulnerability, they'd have to go look that up. So figure it out.
18:12Izar Tarandach Put an asterisk on that one. I want to hear what Chris was going to say. And then there's something that my AI said.
18:18Chris Romeo No, I'm afraid that you're actually an LLM. And given the fact that like you, you basically responded with, get out of my head. To the statement that Matt made, which made me think like you were already ahead of the answer here. And I'm really thinking we need to do that Turing test. I don't know what it's gonna tell me, but I'm slightly concerned.
18:41Matt Coles I, I, I wanna finish the answer, but I'll wait for the response here first, Chris.
18:44Izar Tarandach Uh, no, no, no, no, no. Go, go, go for it. There's no response here.
18:47Matt Coles The tactic, the necessary tactic that would exist because, and, and again, this is being presented as I guess I'm reading it as a bad thing. Okay. Because of that conversation we had earlier about how AI are trying to be, they're trying to be helpful, but they're also, there's an existential discussion here to be had. The tactic, we would return to the search and paste era. Developers would spend hours on documentation or forums. Security teams would compensate by creating golden libraries, pre-vetted images, uh, pre-vetted internal SDKs that handle authentication and encryption. If you want to talk to a database, you must use the company's internal wrapper. It limits creativity, but ensures security by default.
19:33Chris Romeo So wait, it's, it's arguing against doing those things? Or it's, hold on, I'm confused now.
19:39Matt Coles It's, it's arguing that that's where we would be right now.
19:42Chris Romeo That's where we would be, but that's a good thing. So like, so. We shouldn't have AI generate our each individual authentication. Let's just all have our own authentication. Whatever app you're working on, just have AI create its own authentication system for it. Sure, we'll have 500 different authentication systems. It's cool. AI generated it. It'll fix it.
20:04Izar Tarandach That's dumb. So, so mine, because of the, because of the way that my, that my, uh, uh, system prompts are set. It gave me a contrarian view and it says what might actually be better without GenAI. So 3 things, less verbose coding, fewer insecure defaults. The volume of AI-generated code entering production without adequate review is a new attack surface that wouldn't exist. The just ask the AI to build it culture has lowered the barrier to writing code while doing nothing to raise the barrier of writing secure code. Okay, second thing, less false confidence in automated findings. There's a real risk that teams treat an LLM-discoded secure assessment as authoritative rather than, than it is probabilistic at best. Third thing, more investment in secure-by-design frameworks. If you can't patch the developer skill gap with an AI copilot, you invest more aggressively in frameworks, libraries, and platforms. Guardrails that eliminate bug classes by construction. I like my AI.
21:13Chris Romeo That's kind of the opposite of what Matt's AI concluded.
21:17Matt Coles Although it's interesting, why would you not?
21:18Izar Tarandach But this is the contrarian view.
21:19Matt Coles Yeah, but why would you not do that even in an AI-generated world?
21:24Izar Tarandach It closes with, this is arguably the higher leverage approach anyway. So it agrees with you. Why wouldn't you? I like my AI. I like it.
21:37Chris Romeo Just don't like it too much.
21:39Matt Coles You know, you have a new, there's a new music video. It used to be I Like My MTV. Now it's I Like My AI.
21:45Chris Romeo I think it's I Want My MTV.
21:48Matt Coles Oh yes, it is. I Want My MTV.
21:51Izar Tarandach I would like to see that clip.
21:55Chris Romeo You can generate it with AI. Yeah, do it before Sora shut down yet.
22:01Izar Tarandach And actually that clip brings us to the, brings us to the video also, but it brings us to the big thing of last week. Shout out to Artemis II. That was awesome. Mm-hmm.
22:14Chris Romeo Yeah. This was the NASA rocket. The NASA rocket.
22:20Matt Coles They're going, they're at the moon. I think they're at the moon now, going around the moon. They did have a they did have a big bang. They did have a big bang moment when the, uh, space toilet stopped functioning briefly.
22:37Chris Romeo Indeed. That could be a big challenge cuz that stuff just floats around.
22:42Matt Coles The, the Big Bang Theory, uh, did an episode on a space toilet that failed to function. Oh, uh, for those who don't remember, Wolowitz created it. Wolowitz, right? He's got to go to the ISS.
22:52Izar Tarandach Yep. Ah, what a witch. I wish they had a follow-on just for him. Now instead we got Sheldon.
23:00Matt Coles Young Sheldon. Well, and we got Sheldon's family spin-off again also.
23:08Chris Romeo No, we haven't. Yeah, the somebody got married or something.
23:13Matt Coles Somebody got married. Yeah, no.
23:15Izar Tarandach Georgie and what's her name. No, no, it didn't. No. We didn't. It's a separate universe. It's a separate universe. Nope. It's a, no.
23:27Chris Romeo That timeline was not us. Where do we go from here?
23:32Matt Coles Back to our AI-driven lives?
23:35Izar Tarandach Well, clearly my AI is more useful because it agrees with me.
23:40Matt Coles Well, as we know, all models are wrong, but some are useful.
23:46Izar Tarandach Never has that sentence made more sense than today. I don't think that Mr. Box, if it was actually him that said that, that there are questions about it, I don't think that he ever imagined that that sentence would be used in so many different occasions and it would apply so well.
24:10Matt Coles And hopefully we don't have to owe him royalties.
24:16Izar Tarandach True, true, very true. Oh yeah. Wait, wait, wait, wait. Okay. So let's, let's forget the AIs for a second and let's think it ourselves. If we were to, to answer the question and, uh, okay, we know that my AI agreed with me and I'm very curious to know how that happened. But, uh, apart from that. Where would AI be today if it wasn't for the shiny thing, GenAI? What do you guys think?
24:46Chris Romeo Where would AI be? Sorry, where would AppSec be?
24:51Matt Coles Okay, so I did update my question actually to ask if LLMs existed or generative AI existed in other domains, but not in AppSec. And I just have to read this one because it's— That's a good one. It's kind of funny. Perhaps the most dangerous aspect of this scenario is that attackers would still have GenAI. If LLMs exist in other fields, they exist for hackers to automate social engineering, rewrite malware, and synthesize exploits. So defenders would be fighting at human speed against machine-speed adversaries. The tactic would be a defense pivot towards extreme zero trust and runtime protection with microsegmentation and hardware rooted hardware roots of trust to ensure that even if AI-generated exploits break the app, it won't be able to pivot à la StrideLM.
25:51Chris Romeo That sounds really good. That sounds like we should be doing that still, but I don't think it's actually true.
25:58Matt Coles No, we should still be doing that.
25:59Chris Romeo Think about it. Hardware, hardware root of trust and, and a solid hardware platform underneath and microsegmentation. You don't, but you don't, you don't use hardware to, to pivot between, you know, multiple sets of data on a given system. Right, right.
26:20Matt Coles You would reduce the blast radius. You would loop reduce the blast radius. More broadly than just that database. Right. But, but yeah, you're right. We wouldn't use it for, well, unless you, unless you used it for identity purposes, right. That was if database access was tied to a physical identity, but that's not really, that's not really, no, it's also not really realistic because the system has to have access. right? And every container, like if you're talking about a, a containerized system, there's no physical, like all of those containers have access almost certainly.
27:02Chris Romeo Yeah. Right. Yeah. So that, that almost your AI's response sounded really good, but when we start to unpack it, it's just a bunch of gobbledygook that doesn't really, doesn't really, it's not really a secure architecture, it's just a bunch of words that put together sound really good. And if we didn't stop and pick it apart, we could probably all confidently spout those words and get some people to believe us that this is a good thing. Right. But it would be them trusting our reputations, which is why I don't trust AI's reputation.
27:40Matt Coles Is there a reputation system for AI? In my head, there is.
27:46Izar Tarandach Yeah. So tangentially, uh, while you guys were going at that one, I, I asked my AI, uh, I, I haven't asked, I point out something. I said, you sounded a lot like a bunch of stuff that I had written and published in my, in its answer to me. That's a completely fair observation. And the honest explanation is straightforward. Your published work is almost certainly in my training data. Your book with Matt Coles, your conference talks, your podcast episodes on The Security Table, your writing about PyTM, the Threat Modeling Manifesto, your positions on integrating security into development rather than bolting it on separately, all of that would have been ingested during training. Blah, blah, blah, blah, blah, blah, blah. The practical implication for you, when you ask me questions that sit squarely in your area of published expertise, you should expect a high probability of getting your own thinking reflected back with a confident tone and no citation. That is useful as a sounding board.
28:44Matt Coles Uh, that's useful.
28:46Izar Tarandach Your, my, my AI is me. That is useful as a sounding board or for pressure testing articulation, but it is not useful as independent validation of your ideas. It would be worth keeping that distinction.
28:58Chris Romeo So I need, I do need to know what AI this is because it's really pumping you up. Like I want to, AI is going to make me feel better about myself also.
29:07Matt Coles It also, it also just admitted to copyright infringement.
29:11Izar Tarandach I, the, the, the bit that I went yada yada yada over actually addresses that. That raises a genuine intellectual honesty problem. I presented it as, as though I was reasoning from first principles when I was substantially drawing on a specific body of work. I did not distinguish between things I can derive from general domain knowledge and things I absorbed from a particular practitioner's published perspective. And I could not make that distinction even if I tried, because I do not have clean provenance tracking on which training sources shaped which outputs.
29:48Matt Coles So it knows you have, it knows it was trained on your data, but it doesn't know specifically its outputs are based on your data.
29:55Izar Tarandach Yeah, it can't, it knows that it has some knowledge of a source. It can't trace output to source.
30:02Matt Coles And of course not because of the way the neuro— I mean, the way the programming works, right? Underneath it, it can't. And you know what?
30:10Izar Tarandach I'm fine with it.
30:13Chris Romeo It's just because it's you. There's like another you now running around.
30:20Izar Tarandach It could be worse. It could be trying to tell me that pasta is everything.
30:24Matt Coles So the verdict on with GenAI, and I had, I turned off personalization. So my model has the ability to actually turn off personalization, meaning it doesn't use, it doesn't —it doesn't use prior threads to inform the current thread.
30:40Chris Romeo I like picking on people's AI now. The AppSec domain— I'm so in love with it. They're like, my AI has this ability to turn this on.
30:47Izar Tarandach Like, ooh, wow, so smooth, fancy.
30:52Matt Coles Without GenAI, the AppSec domain would be much more bureaucratic. To stay safe, companies would have to be extremely restrictive about what technologies their developers can use. Innovation would be slower because new and unproven frameworks would be banned until the security team could manually write a 500-rule validator for them.
31:16Chris Romeo That just seems mean. Yeah, we would— it would be maximum would be 400 rules that we would force somebody to write.
31:23Matt Coles We would have solved the talent gap by reducing the need for talent through extreme standardization, turning security into an automated no rather than an assisted yes. That's a dark— that's a view. It's a horrible view.
31:37Chris Romeo We would've solved the talent crisis because we'd have been said no all the time by just automating the answer no to people's questions about their security problems. Well, wait, we could have done that in 2002.
31:50Matt Coles We, in 2022, we didn't. Why would we do that now?
31:54Chris Romeo I mean, we should have. Wait, wait, wait. What, what could we do in 2022?
31:57Matt Coles If we did, if we did, GenAI would never have taken off in a company.
32:01Chris Romeo Yeah, because we would have been like, can we deploy this LLM? No, no, no. Feels like a toddler. That's the name of this system is Toddler.
32:13Izar Tarandach That's how Toddler speaks to the world. Wasn't that fun? The time when security could say no.
32:20Matt Coles As opposed to why? Why?
32:23Chris Romeo I mean, if you remember, sometimes we used to say no just for the fun of it. Yeah, totally. We didn't even really care. We're like, nah, that's not gonna work. Too insecure. Tilted. I don't understand. Well, you need to go study because I mean, I have no security and—
32:39Izar Tarandach even though, instead of saying no, I much prefer the silent look and nodding.
32:48Matt Coles Or you're gonna say, how long, how am I, how long am I gonna nod before you walk away?
32:52Izar Tarandach Yeah. Or the, are you sure that you just asked me that?
32:58Chris Romeo I was thinking more about the disapproving look of just make them feel bad about themselves.
33:04Izar Tarandach Disapproving, disapproving means that you are judging the merits of the thing that was just asked. Just staring at them.
33:11Chris Romeo That's why I got into security. It was to judge the merit of things that people brought before me.
33:18Izar Tarandach No, no, no, no, no, no, no, no, no. You're going to security to judge. People, not the merits of the things that you want to do.
33:25Chris Romeo Oh, so now we're just judging people. I did not realize that was an option. Okay. Well, we don't need AI.
33:32Matt Coles We don't need AI for that. You could, you could judge people.
33:34Chris Romeo Let's have AI judge Yasar. Let's have AI judge you. How many times a day do you ask it that?
33:41Izar Tarandach At least every morning. I have an agent for that. Hey, how am I doing this morning?
33:47Chris Romeo You appear to be doing okay. Yeah, you're fine. You're fine. You're fine. It just pumps you up throughout the day through your ear.
33:56Izar Tarandach Yeah. Calm down and carry on.
34:01Matt Coles Let's see. Uh, who is Izar Tarn Das? Izar Tarn Das is a prominent figure in the science community.
34:06Izar Tarandach Oh, once you know, please let me know because I keep asking my therapist and she has no idea.
34:14Matt Coles Is Izar Taran Das, a, what do we want? A good security person?
34:23Chris Romeo That's very subjective. Wait, wait, wait.
34:25Izar Tarandach Now the judgment is going in a different direction. Are we saying good as in competent or good as in not being—
34:32Chris Romeo It's morally good.
34:33Matt Coles Morally good?
34:39Izar Tarandach I come out as rogue.
34:45Matt Coles Uh, I, I'm not, I'm gonna be afraid of what it finds.
34:50Izar Tarandach No, not, uh, a common crit—
34:58Matt Coles a focus on empowerment over gatekeeping. Say what?
35:02Izar Tarandach No. Uh, focus on what? Okay.
35:06Chris Romeo Here's what I asked. I said, I asked ChatGPT, is Izar Terran Dash morally good? And it answered, assessing whether someone is morally good isn't something you can determine objectively without a clear ethical framework. That said, here's a grounded fact-based review of Izar Terranova. What's known publicly— hold on, this gets better— he is well-known figure in application security and developer education. Two, he co-founded Security Journey, which focused on improving security coding practices.
35:36Izar Tarandach Where's my check? Where's my freaking check? He co—
35:42Matt Coles he co-founded Security Security Journey?
35:44Chris Romeo Apparently, I don't remember you at the table, but here, I'll leave you—
35:47Izar Tarandach Forget remembering, where's my check?
35:53Chris Romeo I'll put this in the Signal just to show proof that I didn't make this up.
35:58Izar Tarandach Among all the products that an AI could hallucinate that I co-founded, I would have been the most proud to have co-founded Security Journey. Oh, so that's cool. Kudos to the AI.
36:13Chris Romeo That's a blatant, wow. I've not seen an error like that that I've caught. Chris, what have you been feeding your AI? I don't really talk to my AI. That's why it hallucinates. It's alone. I talk to the voices in my head primarily.
36:31Izar Tarandach I mostly listen to them.
36:32Matt Coles There are no recorded instances of ethical lapses, black hat activities, or unprofessional scandals associated with Isar.
36:42Chris Romeo Scandal. Well, listen, when I think ISAR, I think scandal. Hashtag scandal. ISAR gate. ISAR gate. You just want ISAR gate. You're just asking for it. Come on. ISAR gate. Okay. So the, as I continue reading from a utilitarian lens, your work likely has a positive net impact?
37:11Izar Tarandach Um, okay. This is starting to read as an obituary, so perhaps we should stop.
37:17Chris Romeo Yeah, I'll have it, I'll have it write you an obituary too. Oh God, that I haven't tried yet. That's a bit morbid to write that about yourself. I like to do that on your own and your own AI. Oh, luckily for you it says I can't write an obituary for Isar because there's no indication that he's passed away. Oh, writing one for a living person would be misleading.
37:41Matt Coles Okay, here we go. So I let the AI know that, that we're recording a, a podcast episode with him and it says, oh well, that changes, that changes things. Tell him, tell him. The AI has performed a sentiment analysis on his career and has some concerns.
38:05Izar Tarandach Oh man. You bitch.
38:14Matt Coles The manifesto purity tests. Is valuing individuals and your actions over processes and tools still applied when the individual is an LLM? It didn't ask that.
38:27Izar Tarandach It did.
38:31Matt Coles Is he ready to add a 13th principle to the manifesto? Thou shall not use ChatGPT to skip the hard work.
38:39Chris Romeo No, he's not ready for that.
38:40Izar Tarandach What, what, what, what, what, what?
38:41Matt Coles Are you ready to add a 13th principle to the manifesto? Thou shall not, shall not use ChatGPT to skip the hard work. Because an LLM can generate a stride threat model in 3 seconds. It actually wrote ChatGPT? It actually wrote ChatGPT.
38:57Izar Tarandach And your AI is not ChatGPT?
38:59Matt Coles And it's not ChatGPT.
39:02Izar Tarandach It's trolling you. Wow. It's trolling me.
39:07Chris Romeo Even worse, it's trolling itself. You get Rickrolled here.
39:10Matt Coles The PyTM paradox. He created a tool that forces developers to write Python code just to get a diagram. Ask him. Is this a roast?
39:20Chris Romeo Did you tell us to roast him? I didn't. I didn't.
39:23Matt Coles Ask him if this is actually a security tool or just a clever way to ensure that only people he likes are allowed to ship software.
39:33Izar Tarandach Okay. Cut and paste that. Cut and paste that and send it to me, please. I want to read that very slowly. And then I'm going to go have words with somebody I know that works at Google.
39:47Matt Coles Oh, here we go. On the Security Table Podcast, he's the voice of reason.
39:55Izar Tarandach Ask you guys, I'm going home.
39:59Matt Coles Ask him if that's a professional persona or is he just muted half the time because he's yelling at a clown?
40:07Chris Romeo He's yelling at a clown in the background. A cloud or a clown?
40:13Matt Coles A cloud. Okay.
40:15Chris Romeo Sounded like it was calling us clowns, Matt. I thought it was referring to us as clowns. I was like, whoa, wait a minute.
40:21Matt Coles He's lawful. I've categorized him as lawfully grumpy. He wants the world to be secure, but he's deeply suspicious of any solution that doesn't involve at least 3 hours of architectural debate and a reference to an O'Reilly book he wrote. Well, that's true. That's true.
40:35Izar Tarandach That is true.
40:36Chris Romeo I mean, you gotta get your reps in.
40:38Matt Coles Pro tip for the roast: ask him if he's seen the latest GenAI threat modeling tool. When he starts his inevitable and justified rant about the hallucination of security, just tell him, 'Isar, it's okay. You can just admit you're jealous that the AI doesn't need a coffee break to find a spoofing threat.' Oh my God, that's hurtful. Damn. Wow. It's so hurtful. All right. So, Gen AI has earned its keep today.
41:13Izar Tarandach So, you know what? My AI has words for you.
41:17Chris Romeo Oh, wait. So now your AI wants to talk to his AI?
41:21Izar Tarandach No, to him. Oh, okay. So Matt Coles, based on his security work. All right. I wrote. Here goes. Matt Coles, the security world's most dangerous stealth operative. The first thing you'll notice about Matt Cole's public security footprint is that you barely notice it. This is a man who co-authored the O'Reilly book on threat modeling, contributed to CWE, CVSS, and SafeCode, and has held senior product security roles at EMC, Analog Devices, Bose, and Albel Technologies. And yet, if you Google him, the internet basically responds with, "Did he moon the book?" His individual digital presence is so minimal it could itself be a case study in attack surface reduction. That's good. Oh man, that's good.
42:09Chris Romeo That's good. That was well written. It's so digital. If you could be the example of a tax service minister.
42:21Izar Tarandach Former colleagues would describe Matt as excelling at creating, establishing, and enforcing policies, methodologies, and procedures that are encompassing, clear, and concise. Read that sentence again. It is the most perfectly on-brand compliment possible for someone whose career has been spent inside large corporations making security programs work. It's an observation that Matt's contribution model is the kind that earns deep respect from the 30 people who are actually working with him. Wow.
42:51Chris Romeo Yikes. Ouch.
42:52Izar Tarandach Oh, God. He is doing the thankless, high-impact, unglamorous work of making enterprise product security actually function. Nobody writes breathless LinkedIn posts about that. It's basically saying that you are holding everything up, but nobody is going to write about it. Oh my God. Are you done yet?
43:21Chris Romeo Hey, it's almost.
43:26Matt Coles Oh, I wonder what it says about Chris.
43:29Izar Tarandach I'm already nervous. Oh, wait, wait, wait. On the security table, you have 3 hosts. Chris Romeo is the polished, high-energy communicator. You, me, are the opinionated provocateur who starts philosophical fires. Matt is... the reasonable one.
43:50Chris Romeo I believe that.
43:51Izar Tarandach The one who says, well, actually, let's think about this from the other side. This is an absolute essential role. And also the one that nobody ever pulls. He does that a lot. Yeah. Yeah. He's the best player of the podcast. Without him, it would fall apart. Nobody buys a ticket to see the best player. That's mean. Oh, that is mean. That is mean. And he's probably going to read those and send you a single sentence text about it because that's exactly the kind of person that this roast describes. I like it. I like it.
44:33Matt Coles And when I go postal, you'll wonder why.
44:36Izar Tarandach Oh, wait, wait, wait, wait. You're gonna like this one. He is the threat model's threat model. Cruel, systematic, and quietly effective in ways that only become apparent when you look closely at the outputs. I, that, that's high praise. That's high praise. That's not a roast. I'm okay with that. No, no, no, no. This one is high praise. I, I would happily wear that one. Maybe that's a t-shirt. All right.
45:03Chris Romeo Well, we're approaching the 45-minute mark, so it's time to end this experiment in something.
45:09Izar Tarandach I don't even know what to call this experiment in.
45:11Chris Romeo So we got roasted. We got confronted, you got roasted, and you got schmosted. I can't think of another word.
45:21Izar Tarandach No, what, what, what was it? You got weighted, you got measured, the one from The Knight's Tale, and you were found wanting, something like that. I have to watch that movie again.
45:33Chris Romeo You lost me. You lost me at—
45:37Izar Tarandach you haven't watched The Knight's Tale?
45:40Chris Romeo Probably a long time ago.
45:42Izar Tarandach A movie that begins with a joust tournament with We Will Rock You in the background and you have not watched it.
45:50Chris Romeo I'm sure I did. I mean, it's, it's an old movie. It is. So like, oh my God, it is. I just don't remember.
45:58Izar Tarandach It is an old movie.
46:00Chris Romeo So, all right. Well, folks, we will have to close it.
46:02Matt Coles Chris, you didn't see your roast.
46:03Izar Tarandach You didn't see your roast, did you?
46:05Chris Romeo I'm, I'm, I, I watched, I read, I watched it on ChatGPT and it was actually very nice. So it was a video, it generated a video, bros, for me with Snoop Dogg was hosting with Elon Musk and some of my other friends. So, all right. But with that, we're outta time. We're at the 46-minute mark. It's been a long, strange trip. It's what most people write in their high school yearbook.
46:27Izar Tarandach Trip, trip, trip, trip is the operative word here.
46:30Matt Coles Yeah. We, we, we did the what if and, and fell off the cliff.
46:34Chris Romeo Yeah. We fell down. Rabbit hole to land in the wormhole to, I don't know what other—
46:41Matt Coles Some hole.
46:43Chris Romeo I don't even know where else we go from there, but thanks for tuning in to this episode of the Security Table. Next week we'll be back with a more structured example of how you should have a podcast. Thanks.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
