More Cowbell: Security and Speed in Agile
With Chris Romeo, Izar Tarandach, and Matt Coles
We’re diving into the relevance and execution of threat modeling within agile development environments. We dissect the claims, explore the true integration of agile practices with threat modeling, and address the misconceptions and challenges commonly faced.
About this episode
We’re diving into the relevance and execution of threat modeling within agile development environments. We dissect the claims, explore the true integration of agile practices with threat modeling, and address the misconceptions and challenges commonly faced. Check out the episode to find out if threat modeling is indeed slowing down agile processes or if it can be seamlessly integrated for better security outcomes.
The Problem With Threat Modeling in Application Security: Too Slow, Too Theoretical, Not AgileFollow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 49 min conversation
0:09Chris Romeo That's what this podcast needs is more cowbell, right?
0:14Izar Tarandach Everything needs more cowbell.
0:18Matt Coles You don't have a sound effects panel in here somewhere?
0:21Izar Tarandach That's what I'm getting.
0:22Chris Romeo She's downloading it, folks. Just hit— just stand by. Isar is collecting his sound.
0:29Izar Tarandach Wait, it's gonna be funny.
0:31Chris Romeo He's having, he's bot coding it right now. AI is creating an app.
0:36Matt Coles He's, he's gonna need somebody to clean. He's gonna need somebody to clean that up though.
0:39Izar Tarandach So my AI is on it.
0:43Chris Romeo Have you heard anybody say that yet in conversation?
0:46Izar Tarandach What?
0:46Matt Coles What?
0:47Chris Romeo My AI is on it.
0:48Izar Tarandach No, but I have told people to have their agent talk to my agent.
0:53Chris Romeo And it's an entirely different thing. It's not a Hollywood thing.
0:57Izar Tarandach No, no, no, no, no, no, not that kind of agent. Not that kind of talent.
1:01Chris Romeo Right. You guys don't have agents for the podcast here?
1:04Izar Tarandach Wait, that's an option?
1:06Chris Romeo That's why your contracts were so easy to negotiate at zero.
1:11Izar Tarandach That could be an option.
1:14Chris Romeo Yeah, that thing I tricked you into signing.
1:16Izar Tarandach I thought the hot dogs were free.
1:20Chris Romeo That was a 100-year deal.
1:24Matt Coles I'm not just a podcast host, I'm also a sponsor.
1:26Chris Romeo That's right. Oh, well, hey folks, you just joined us. What seems like an existing conversation where 3 people were riding in the car together for a really long time and it almost ran out of things to talk about. But welcome to this. Oh wow, I heard something coming in there, but he'll stop.
1:44Izar Tarandach We have cowbell.
1:46Chris Romeo Hit the cowbell. You want to do it?
1:48Izar Tarandach No, that's not a good cowbell.
1:50Chris Romeo It's not bad, but I mean, it doesn't, it doesn't really hit the right rhythm that we're looking for, you know.
1:56Matt Coles Ain't got no big time.
1:58Chris Romeo Well, I'm hoping Christopher Walken will accept our request to be on the show at some point in the future.
2:05Matt Coles We need some star power.
2:07Chris Romeo Yeah, other than Izar. I mean, we've got Izar.
2:11Izar Tarandach I'm getting old.
2:13Chris Romeo Yeah, it's true. He is. He's got like a couple more gray hairs from last week. Yeah, well, I guess we should officially introduce ourselves after a few minutes of discussion on the Security Table. I'm Chris Romeo, joined by Izar Tarandash, Matt Coles. Welcome to the Security Table. So let me set up what we're about to talk about here. There's a little article that got posted on LinkedIn. And it's called "The Problem with Threat Modeling and Application Security: Too Slow, Too Theoretical, Not Agile." And so I thought we would kind of walk through this and maybe unpack it a little bit, maybe offer a little bit of commentary along the way. But I guess, what's y'all's reaction to this at a high level when you hear this, you see the title stated as is?
3:10Izar Tarandach I, I, I think that it's a very proper comment. I think that it's very well placed. I think that it's very well, uh, developed. If this was 2001, the gloves have been removed.
3:28Matt Coles Love it. No, seriously.
3:38Izar Tarandach I mean, not even 2000, I'll give a couple more years. Like I'll say, yeah, 2000, I don't know, 17, 18. But I think that, you know, things have evolved a bit, especially in the last few years. Approaches have changed, tools have gotten better, people have gotten better, the acceptance of threat modeling has gotten better. So, I would say that this is an argument that is valid only if you want it to be valid.
4:07Chris Romeo So it's also, it's also some interesting clickbait.
4:11Matt Coles It, it, yes, it is to get people fired up.
4:13Chris Romeo Like, I mean, I've created articles like this too, just to get people riled up. It's, you know, it's fun.
4:17Izar Tarandach Yeah.
4:18Matt Coles Now I'm going to give him a little bit of benefit of the doubt. I want to hate this article, but I'll give a little bit of benefit of the doubt here. If you think about it, threat modeling is, does require manual effort today for the most part, right? AI threat modeling is still in this, in its infancy. We haven't quite figured out how to make it effective. And, or, or, or it happens really too late, right? Because you look at code and if you're looking at code, you're not looking at design. And so you really want, you really wanna make this happen early, well, what are you working with? Now you're working with, you know, large text documents and functional specs and things like that. But by and large, threat modeling is done on a whiteboard or using the Microsoft Threat Modeling Tool or using PyTM, where some manual effort is required. Now, does that make it too slow for an agile development effort? I, thinking on that. And I think we have, uh, you know, good evidence in the industry overall to, to, that indicates that, but there is manual effort involved in threat modeling. In fact, the Threat Modeling Manifesto almost certainly requires it in some place, right? Not necessarily model construction, but certainly model interpretation and dissemination of results.
5:48Izar Tarandach So I agree with everything that you said, but I want to turn this a bit on its head and ask a different question. If it is slow, why is it so bad to slow things down every now and then?
6:04Matt Coles I mean, isn't velocity everything?
6:07Izar Tarandach Uh, I don't know, perhaps, uh, going to market faster is is everything, but I think that we are seeing the blast now from this fastest fade of creating prototypes and turning them into products and just releasing them in the wild. And if we take that as the extreme of going too fast, and we say that certain threat modeling methodologies are the extreme of going too slow, isn't there a golden thing in the middle that says, you know what? It's just the right amount of slow. It's just the right amount of stop and think before you do and regret. And I think that it could be said, it should be true actually, that threat modeling is the moment when you stop and think before you go out and regret.
6:56Matt Coles Well, and you don't even have to stop and think. You can run the activities in parallel. True. You just want to Towards the, towards that decision point of where you say, is this ready to go out the door? Include threat modeling as part of that, is it ready?
7:15Izar Tarandach But then you get into this, is this threat modeling or is this an architectural review thing, right? We want to do it before so that we can stop big things from happening rather than saying we identify big things and we want to fix them in the next. Version. And I know that what I'm saying is completely unattainable nowadays, but it happens sometimes in many places. That's why I'm saying again, it can be slow, it can be not agile if you decide to make it so. But if you decide to make it faster and agile and evolutionary, the tools are there to do it. And by tools, I mean the processes, the people, The techniques, the tools, that there is a huge body of work out there saying that, yes, you can do this thing. Bearably slow.
8:10Matt Coles Well, I kind of want to unpack that a little bit. I know Chris is a huge advocate. I think, Chris, you're a huge advocate of this, of agile development practices and speed. But what is agile development really? Nobody, can somebody explain that to me? Because I know what an agile development process is, but what does it mean to be agile? Because that might help us understand if threat modeling is agile or not.
8:43Izar Tarandach It's building the car as you careen down the hill.
8:50Chris Romeo Taking small chunks, taking small bites at a time. How do you eat an elephant? You know, small, tiny bites at a— over a period of time. So that's, that's what, how I think about it because, and, and when you contrast it with where we came from, we went in a waterfall model, we went through this big, let's plan, do requirements. Architect, code, test, and then 2 or 3 years from now, we'll see how we did by stacking it all together. Agile is the answer to what everything that's wrong with what we were doing in the waterfall world, which is let's make things run. Things should be, we should be able to run whatever it is that we're working on at every, whatever our sprint interval is, you know, every 2 weeks, 3 weeks, 6 weeks, whatever. When we put all the, when all the code comes back together, it has to work. For us to be successful.
9:48Matt Coles And so, how do you design in that agile process?
9:53Chris Romeo In my world, or my view of the world, I have adopted what Izar Tarendas started saying about 10 or plus years ago, which is threat model every user story. Because I think it's the logical smallest unit that we can engage with that still is attainable, but yet is not too big where people can get lost or they can get, or they can just miss whole classes of threats because the scope was just too complicated. So I'm a threat model user story guy.
10:29Matt Coles So who does, I'm walking you through a scenario here, obviously towards a goal. Who does the threat modeling of that user story? Is it the developer who picks up that story, or is it the person who creates the story or somebody else?
10:47Chris Romeo I mean, in a perfect world where I'm aiming organizations towards is the developer does that modeling process.
10:59Matt Coles Okay.
11:00Chris Romeo In reality today, I see some, I see very few where the developers have completely embraced or taken on the threat modeling steps. I see some where the champions have gotten that call from the security team. I see others where the security team is still working on it, they're still doing the threat modeling. But for me, the only way that you can scale this, and to, you know, some of Sean's points about, you know, it's too slow, it doesn't meld with organizational processes, yeah, if you don't get it to the point where the developers are doing it, When you have the developers doing it, now you've got enough resources. And once they start to get experience in doing threat modeling, it shouldn't take multiple days to do a threat model for a user story. It should take an hour or two if you got the right tooling that rock drive, you know, walks you through the process and helps you fill in the knowledge gaps that you have, for example.
11:58Matt Coles And is it a subtask then as part of a story that you would do threat modeling?
12:04Chris Romeo I see it as more as part of the process versus a subtask on the story itself, because it's kind of a meta thing, right? Like, if you have a subtask of something on the thing, it's not as important as something that sits above and equal to the thing.
12:21Matt Coles So it's part of understanding the story that has to be implemented.
12:25Chris Romeo Yeah, which is, which is really valuable for developers to do. If they actually understand what they're supposed to be building versus just starting to code, which I mean, some people do that. They're just like, let me code something, but you don't understand. And so, yeah, that's another hidden benefit of threat modeling, right? Is that it causes you to have to pick something apart and really understand what it's calling for.
12:49Matt Coles And if you're looking at two stories and you have two developers working on each, one on each story, and they're doing their own threat modeling, and there's a, a connection between them. There's some facility for making that connection, right? Yeah.
13:04Chris Romeo Yeah.
13:05Matt Coles So, and that establishes a, a broader understanding across the development team.
13:09Izar Tarandach And, and, and that's why in, uh, continuous threat modeling, even though the developers are tasked with doing that threat model, there is the idea of a threat model curator who gets the results of those, uh, uh, story threat models. And finds those links that you mentioned. So that's the place where you have like a bubble up and you get the commonalities of two stories and you say, oh, now I have an opportunity here to design common solutions for these mitigations, right? But, uh, I, I, I really liked the way that, uh, Chris addressed it, but I've started looking at it at, uh, from the point of view of responsibility. right? We'd like to describe the eureka moment when the architect is describing the design in a session, in a threat modeling session. But then the new programmer in the team, the most junior guy in the room, or lady, raises their hand and says, yeah, that was the design, but because of limitations and this and that and the other one, we had to implement it actually this way, right? So the person who's going to do that decision to change the design because of the limitations or whatever other constraints, that person is actually taking responsibility for a change in design that is not formal. It's not documented up the chain to be reflected in the design document if it exists or not. So if that is the person who's responsible for the threat model for every story, what you get in the end is that The threat model becomes the source of truth about the system that's being threat modeled. So you have a transference of responsibility from the architects that had the big idea and suggested how it would be implemented to the person who's actually implementing and is hitting, getting hit by reality in the face and having to dance with that thing until it works securely, hopefully. And that's what actually gets documented, right? So I've been looking at it mostly from the point of view of responsibility. And what's interesting to me here is that in the last few years, been thinking about privacy threat modeling. And in there, I see an inversion. I see the responsibility being on the person who actually creates the story because they are the ones who have the bigger view of the whole system and can think about privacy threats that somebody closer to the code probably would not be able to.
15:41Matt Coles Because you need a more systemic view for privacy than you do for, for security. Yes. So I have a few more questions and, and just a comment. One might think of maker-checker, uh, kind of discussions here on your, uh, the person who's re-architecting the thing at the local level is going to also threat model at the local level. But that may also be just taking responsibility, which is, which is good for each developer if they have the skills to do it, as opposed to if they don't have the skills, they're, they're likely to just pass it without really thinking about it.
16:19Izar Tarandach But that, that's where the, that's where the thing with, uh, uh, moving fast comes, comes in, right? To, to paraphrase the, the. The common wisdom, you should go fast, but only as fast as you can. And, or as you should. And, well, no.
16:43Matt Coles So many questions on that, but go ahead.
16:45Chris Romeo No, no.
16:46Izar Tarandach What I mean by that is you want to go fast, sure, go fast, but you have to recognize when you have to stop for air, right? Just keep going at, at a maximum speed and you're going to hit the wall at maximum speed. You have to recognize that sometimes you have to stop and get your bearings or get some air or something like that. And that's the moment when you realize that, hey, I need to do something in here that perhaps I don't know enough. And now I need to have some education. Now I need to have some training. Now I have to talk to someone. Now I have to ask someone. Now I have to have a dialogue with an LLM, right? Reach out to a book, reach out to an LLM, reach out to a person. But at some point, that slowing down is what's going to enable you to down the road go much faster.
17:40Matt Coles So if I understand correctly, in Agile, there's a notion of a spike.
17:48Izar Tarandach The sprint spike.
17:49Matt Coles Sprint spike.
17:50Izar Tarandach Yeah.
17:51Matt Coles Is a spike a good place to do threat modeling?
17:54Izar Tarandach Microsoft used to think that, yes, it was, because that was their approach to, uh, uh, agile threat modeling, right? They would say, oh, you, you have your normal agile sprints, and every now and then you have a security spike where you stop everything and you only do security, you only do threat modeling, you only do mitigation and all that good stuff. And still it, it, it left gaps, right? Why? Because now you are playing again with the context switching. You're telling people work for a month on as many stories as you can, spend as many story points as you can. And then we are going to stop and say, oh, remember the story that you did 6 ago? Let's talk about threat modeling that night, or let's talk about the one that you're going to do in a month. And people can only context switch so much, even if they live deep inside one system, right? So I find that separating people from the threat modeling with a time delta and ask them to be as effective as they can. Is at most aspirational. I don't even know if Microsoft still does that, by the way.
19:17Matt Coles What are your thoughts on this, having run a few, you know, agile development shops? What's your thoughts on that?
19:24Chris Romeo As far as the spike, you mean?
19:26Matt Coles Yeah, the value of spike in threat modeling.
19:29Chris Romeo Yeah, I mean, I always, the way that the teams that I've worked with used spikes were different. It was more of a research-style task. So, when they weren't ready to design or fully design and code something, they would use the spike as a way to go do some research into various libraries or frameworks or methodologies that they could use to solve the problem that we had without actually building it. It was more like a research thing. So, I think the The threat modeling process has to be tied directly to the user story, because if you think about it, that is the unit of work for modern development teams now. So, there's a Jira ticket in a lot of different places, and there's, you know, other tools other than Jira, Azure DevOps, and other things, but Jira is kind of the unique, the standard thing that we say, right? But each user story has a Jira ticket associated with it, the developer grabs the Jira ticket, they comment on it, they work it, they change statuses. So why not put the threat model right in the Jira ticket if that's what— if you're using Jira? So it's— so that's why I land on let's use the user story to house the threat modeling context and records of what we've done because that is the standard unit of measurement for development teams today.
20:57Matt Coles And just to bring this conversation into the newest new realm of, of agentic and vibe coding, or AI tool-assisted coding, that still holds whether you're talking about a human individual pulling a ticket, a story off of the, off the backlog, or an agent pulling a story off the backlog, right? They have to think of, they have to understand what the story means. They have to think about what it, what it, how it would translate to code, at which point they can do threat modeling, at which point they can look at that story and say, this is going to affect X, Y, and Z parts of a system, or take input or send output, and can do threat modeling, correct?
21:44Izar Tarandach I don't know if they can do threat modeling. I, I, I have in the past few months played with it a bit, and I, I have gotten good value on, uh, So, sorry, I wasn't talking about the agent itself.
21:54Matt Coles I was talking about the process, the thought process of pulling the strings.
21:57Izar Tarandach Oh, yeah, yeah, yeah, yeah, yeah, yeah, yeah. That there, there definitely is something there.
22:03Matt Coles So if a human is using a coding assistant, before they even ask the code to be written, or maybe while they have the code starting to be written, they could be doing threat modeling.
22:16Izar Tarandach Who's they in this scenario?
22:17Matt Coles The human. Initiating the process of pulling a story off the backlog?
22:24Izar Tarandach There, I'm having a bit of a difficulty seeing good things because, and this comes as no criticism, but just an observation of what I've been seeing lately. Those people who are interested in adding security in at that stage, they are basically farming the threat modeling out to the LLM or to the agent, right? So they're saying, this is the story, threat model it for me. And by the way, after you finish that, code accordingly. And, um, I just don't think that LLMs are completely there yet. I don't think that the agents and AI-based tools are in there yet. That's why I so much liked the, the, the Vici approach to infuse AI in the process, but not make AI take ownership of the process. So, could it work? Yes. Is it guaranteed to work every single time? I don't think that that's where we are right now.
23:33Matt Coles So what you're saying is the tooling available in an agile development process may be lacking. It's not a matter of slowing down the developer. It's a matter of the tools are not yet capable.
23:49Izar Tarandach So this is where I was going before, right? I've been playing with inverting the idea and rather than having the LLM answer my questions, have the LLM ask me contextually important questions for the threat modeling, right? So I, I think that the LLM is much more— it's much better equipped to ask those questions than to answer mine because—
24:20Matt Coles so let's unpack that a bit. So sorry, just to— for our, our listeners' benefit here. So you take a user story It's a couple lines of text, some description of what needs to be done, or what, what feature is being implemented, or some task that needs to be performed. You pass it to the LLM, and you have it take a critical look at it, and understand, be able to understand it in some dimensions.
24:48Izar Tarandach So I've been trying to approach this from the point of view of a developer who is not a security person. So leading the LLM down to asking basic questions about what's the security value of the story. Does it even have any? Right? So things like, I need to update some part of the documentation. Hey, do you see any kind of security value in this thing? Well, unless the documentation is about a security function that needs you know, tokens and whatnot and this and that and the other. No, I don't see it. So can you give me a bit more detail about, about what documentation is being updated? So by having, by giving the LLM the leeway to ask me questions back until it thinks that I hate to say until it thinks, but until it has like enough context to go on. Or for me to feel that, for me to think that actually what I'm getting back has given me enough to understand what's happening. I find that that's more useful.
26:04Matt Coles Do you capture the conversation in some log somewhere? Like part of the story?
26:08Izar Tarandach I had a bunch of— I was just trying out things. I wasn't actually doing it as a process. But, uh, I, I got part of them in a, in a LinkedIn article a couple of weeks ago. And, uh, it's, it, you know, it's fun. I, I, I, I compared it to playing tennis with the wall, right? Is it a tennis match? No. But is it going to make you sweat and run around and perhaps work some part of your game? Sure. Interesting.
26:44Chris Romeo Well, let's, we should go back to this article a little bit. Yep.
26:47Matt Coles Because there's a lot of, well, so that line of question was to sort of demonstrate that Agile has built within it the ability to do threat modeling without slowing down the process.
27:00Chris Romeo The two-layered—
27:01Izar Tarandach Yeah. Without slowing down too much. Right. Let's be honest.
27:05Matt Coles Too much. Right.
27:06Izar Tarandach Yeah, it will add, it will add points.
27:10Matt Coles It'll add a point to story. It'll add a point to story if you're doing story points.
27:14Chris Romeo Nothing's free, right?
27:16Izar Tarandach Nothing's free. What you get out of it is much more than losing that point in there. Yeah, let's go back to the article. What else we have in here? Okay.
27:25Matt Coles Oh, no, by the way, while you guys were talking, I've been looking for the Security Boulevard article that is like the linchpin of this, of this of this article that we're looking at, and it doesn't exist. Oh, it gets, it gets, it gets better than that, Matt.
27:38Chris Romeo I've been doing some research myself here. So the Security Boulevard article appears to not exist. So it appears to be an AI hallucination when the article is being generated.
27:49Izar Tarandach Wait, wait, wait, wait. Do you guys exist? Am I imagining this whole thing?
27:54Matt Coles I don't know. Do we? Wait, how would you know?
27:58Izar Tarandach Am I dreaming of electric sheep?
28:01Chris Romeo You can only hope. But then it gets better. So, I had ChatGPT track down the academic paper for me and scour it. And it was interesting because it said, and I quote, the paper does say that threat modeling is not part of daily sprint work for the team studied. However, it does not explicitly say not one team fully integrated threat modeling into their sprints in the strictest possible sense. I did not see a sentence that states that absolutely no team ever did it every sprint. So this is like a restatement of some academic research that nobody— and people do this, they think nobody's ever going to go look at the original source. They haven't met people like us. WolfLifeBlog security people.
28:50Matt Coles Who actually reads articles anymore? Doesn't the LLM automatically do it for you and summarize it?
28:56Chris Romeo That's true.
28:57Izar Tarandach That's true. Yeah, that's it. Who, who reads the, the article nowadays? My LLM.
29:01Chris Romeo Yeah. So I'm not, so I don't know about this Security Boulevard article. I don't know about this academic research now, but I mean, 37% of organizations had a formal documented threat modeling process. That's SANS CTI survey. I can, I can see that. That, that tracks with what I see is about the number of companies that are doing threat, that I think are doing threat modeling in some capacity now. 37% feels like about where we are right now in the industry.
29:29Matt Coles So I wanna, I just stumbled up while searching for the Security Boulevard article that is the Phantom article. I did stumble upon somebody who, and I'll give props to this guy. I put it in the link in the chat for you guys. For making threat modeling work in an agile development process. And it's a 5-step, 5-step process for integrating threat modeling in agile development. Start small. Pick a feature. What can go wrong?
29:59Izar Tarandach I heard that before.
30:01Matt Coles Use simple frameworks like STRIDE or the 4-question framework. Run a 30-minute whiteboard session with the dev and security teams.
30:09Izar Tarandach Oh, that's where it goes wrong. Okay.
30:13Matt Coles Track outcomes and then automate where possible. Yes. Now this was completely, totally written by an AI because of common spelling errors and other grammar errors.
30:24Chris Romeo And the use of emoji style headers.
30:29Matt Coles Okay.
30:29Izar Tarandach But sidestep here.
30:31Matt Coles Well, converting Mark down to LinkedIn.
30:33Izar Tarandach I mean, dude, if you haven't, if you haven't added. To your LLM guiding file or setting, whatever it is, do not freaking use emojis. Okay? Please do yourself a favor. Okay? You are not Egyptian. You are not building pyramids. Stop writing hieroglyphs. Right? That stuff has gone from the world like 3,000 years ago. Do not bring it back. It's not useful. Close parenthesis.
31:05Matt Coles Unicode, the bane of Unicode, the bane of our text, text files.
31:11Izar Tarandach Right? No. If you can't say it in 8 bits, you shouldn't be saying it here.
31:19Matt Coles Right?
31:19Izar Tarandach So here's the thing. Step 1, start small. Awesome. Step 2, use simple frameworks. Awesome. Step 3, collaborate. Run the 30-minute whiteboard session with devs, product, and security. I'm assuming security people. Now, here's where the devil raises its head, right? What is the investment and cost of a 30-minute whiteboard session with Dev, Product, and Security?
31:50Matt Coles Now, just think— It's a spike, isn't it?
31:52Izar Tarandach No, no, just think in terms of hunting these people down. Finding time in their calendars. Okay, finding time in their calendars. Getting everybody up to speed in 30 minutes.
32:07Matt Coles Don't you do this during sprint planning? This is where everyone's like sitting in a room, right?
32:10Izar Tarandach And getting something out of it in 30 minutes. So you were doing so well until you didn't.
32:20Chris Romeo Yeah, but I mean, this is one of the patterns.
32:23Izar Tarandach The plane was flying great until it crashed.
32:25Chris Romeo This is one of the patterns we put in the Threat Modeling Manifesto.
32:29Matt Coles Get everyone together. Common understanding.
32:31Chris Romeo Diverse viewpoints. Diverse viewpoints is what we described it as.
32:35Izar Tarandach Yes, but we weren't specifically talking about agile there. Okay? We're talking about functional. We were talking about getting the most out of it. Now, am I saying—
32:52Matt Coles Why is this not going to work?
32:53Izar Tarandach Because of schedules? This one? Because of the amount of investment that you're going to put into getting those 30 minutes.
33:02Chris Romeo Man, hold on, hold on. Versus what you're going to get out of it. Throw in the flag. Flag has been thrown. These companies, these poor people spend 8 to 9 hours a day in meetings already. So nobody cares about how much it costs for any particular meeting. You can't have meetings from 8 AM until 6 PM.
33:21Izar Tarandach I'm not talking costs in terms of money. I'm thinking me as a security person. Okay.
33:28Matt Coles Yeah.
33:30Izar Tarandach I work in a big corporation. I have a lot of different teams that I have to work with. I can count in the fingers of a very small hand the amount of times that I went into Slack and I said, can we meet? And everybody was available. And 30 minutes later, we had an answer. I'm saying the cost in time, in context switching again, in figuring out who has to be in there, who's available, who isn't. There is a whole thing to get 30 minutes out of people. If you're going to get 30 minutes out of people per feature, then you are multiplying that.
34:18Matt Coles And a feature is a collection of stories, right?
34:21Izar Tarandach Yeah.
34:22Matt Coles Could be. Yeah. I'm a big fan of this actually. I'm a big fan of on a per-feature basis, cuz it's not quite as small as a, as a story. It's more as a collection of things, I guess, to Chris's earlier, you know, to the discussion earlier where the developer may have a local knowledge of the story, but the collection of t— of that team is going to understand the feature. And the feature will give you enough context for a threat model. And if you do sprint planning and you have everyone in the room anyway, then making security and threat modeling as part of that sprint planning, at least to, at least to provide direction to developers, hey, when you're working on your story, make sure you update this model that, that the security team put together, because we did some backlog grooming perhaps before. Before the feature actually came up to be implemented.
35:14Izar Tarandach You know what, you convinced me, but not for general threat modeling. For privacy threat modeling, definitely, because that view of the feature is important in privacy. To be able to see the whole process, to be able to see how everything clicks together and what data is being collected and how and how from UI to backend and back up things happen. For security, I would say that aspirationally it's a great idea. Practically, I, it's hard for me to see it happening.
35:56Chris Romeo I think the scope can grow with a feature. I've seen features with T-shirt sizes associated with them. So like you have an XL feature, which is basically what we would call a subsystem in the old days.
36:12Matt Coles Right.
36:12Chris Romeo Right. Like, so that's why I've always stuck to the user story. Like, I mean, and if someone's like, well, we, we batched together a bunch of user stories because they were similar and they were working towards the same goal. Okay, great. I don't have a problem with that. Right. But you do run the risk of the scope getting too large, and we all know that results in threat models that just don't have the amount of detail that we would like to see.
36:41Matt Coles So there has to be a happy medium. But either way, we're, I think we're still continuing to prove here that agile does not stop you from doing threat modeling. Agile can benefit from threat modeling, and agile can integrate threat modeling. It doesn't slow you down in a meaningful way.
37:00Izar Tarandach And the other way around too. So the threat model can benefit from Agile and Agile can benefit from the threat model, right? Yeah. As you said, the importance is to find the happy medium and that changes from organization to organization, even many times from team to team.
37:17Matt Coles Right. Or over the life of the team, because as we know about Agile, your velocity and your ability to take on work evolves over time.
37:26Izar Tarandach There you go.
37:26Chris Romeo Yeah. Let's, let me address another section of this because in the analyst's weigh-in section, Forrester, I'm just going to paraphrase, Forrester says fewer than 4 in 10 organizations use systematic threat modeling at scale. That tracks with the SANS survey previously. And, and once again, I don't see any reason to not believe that based on my experience. Gartner reports fewer than 1 in 5 enterprises have adopted AI-enabled threat modeling tools. Okay. I can see that. That makes sense too. But none of this makes the argument for why threat modeling doesn't work. This is like, let me just throw a bunch of facts together from various reports that'll make it sound like I'm making an— this isn't part of the argument.
38:12Matt Coles Somebody who doesn't adopt it is not doing it. There's no indication of why they don't do it.
38:17Chris Romeo Yeah, exactly. So, um, so if we keep scanning through here, threat modeling still matters. So now his conclusion is threat modeling still matters.
38:32Matt Coles It's a schizophrenic result here.
38:35Izar Tarandach I don't know. My, my, my thing is that when when we reach the, the conclusion, right? I believe strongly in formal risk management.
38:50Matt Coles You know what? With threat modeling as— with in-depth threat modeling at its core.
38:54Izar Tarandach Yeah. More and more I'm learning to separate the things, the threat modeling from the risk management.
39:01Matt Coles Oh, 100%.
39:03Chris Romeo Different disciplines.
39:04Matt Coles So threat modeling should inform risk management, but not be part of risk management.
39:09Izar Tarandach Yes. And should definitely not be confused with. And many people do.
39:15Matt Coles And so the very next line, actually, is where I think we're— I'm having the most issue with this. For this to work, it must be baked into everyday processes throughout the delivery and operations ecosystem. That's threat intelligence, not threat modeling.
39:31Chris Romeo Hmm.
39:32Izar Tarandach Interesting.
39:32Matt Coles If we're— if we're doing operations, if we're doing delivery and operations, we've already written the thing and are ready to ship it. That's not threat modeling, not in the traditional sense.
39:43Izar Tarandach Wait, that means that every stakeholder team needs to examine and update their operating methods, embedding modeling into their workflows rather than treating it as an additional step.
39:56Matt Coles So make threat modeling part of your delivery, your story process, your definition of done. Wouldn't that slow you down?
40:08Izar Tarandach But how does that work with delivery and operations ecosystem? I'm having a problem catching the whole thought here.
40:19Chris Romeo All that's missing is the use of digital transformation.
40:23Izar Tarandach And blockchain somewhere.
40:25Matt Coles Blockchain.
40:27Chris Romeo Don't forget quantum. Quantum will save the day for threat modeling.
40:31Izar Tarandach These blocks are the chains that tie us to the past. The, the real—
40:36Matt Coles evolving without evolving these processes. Yes, exactly. You were just on that, right? Threat modeling will remain a theoretical exercise instead of the foundation of secure design.
40:47Izar Tarandach So basically he's advocating for threat modeling every feature or story or—
40:55Matt Coles well, or doing a design, doing a, doing an actual design as part of the process. So there's a fundamental problem there. If you can't do threat modeling and you're doing a, you're doing story, story by story where you're building the thing as you go, are you really doing secure design fundamentally? Are you? If you do, where do you do it? You probably do it outside the sprint, by the way, right? You do it as part of your backlog generation, right? Where you define those features and you define those stories and you have a concept in mind of what this thing will look like, right? You don't start with implement this feature and design evolves in the, in In the classic Agile approach, does anybody do a classic Agile, pure Agile, where nothing happens unless it's customer deliverable?
41:58Chris Romeo No. No way. It's not possible.
42:03Izar Tarandach I can't stop thinking about 13, 14 years ago when Matt and I took a class to be certified Scrum Masters.
42:13Chris Romeo Yes.
42:13Matt Coles Oh, so bad.
42:14Izar Tarandach So much. Spent the whole, I think, week, right?
42:21Matt Coles Yeah, it was a week-long class, boot camp or something.
42:24Izar Tarandach Looking at each other and saying, why don't they have anything about security here? We were the only security people in there, right? And we came out with the idea of the security toolbox, which was the root of a lot of the work that we did later on. But I remember clearly, we are sitting in the corner of the table. Because it was one big table with everybody around it, right? And we kept looking at each other and going, but where does security go into here? Right?
42:57Matt Coles Right. Yeah. You pull this thing down, you work on it. And now that at the end, you have something that you can ship.
43:02Izar Tarandach Yeah.
43:03Matt Coles Great. Where do you add like the security features? Where do you add the security defenses?
43:09Izar Tarandach I think that it was a bit of a formative time for both of us because it informed so much that we did afterwards. Both continuous threat modeling and the work that we put in the book and all that. And I think that at some point I must have made a big mistake where I stopped seeing it as something that needs to happen and started looking at it as something that I can't understand how come it doesn't happen.
43:42Matt Coles That's the cynical side of you there, Isar.
43:44Izar Tarandach No, no, no, seriously. No, no, that's probably the naive side of me in there. And the more I spoke with architects and developers and teams, and people would look at me and like shake their hands and say, yeah, that makes perfect sense. But how do we do it? And then came the, they've, these, them, that checklists. And, and, uh, ITM and, and stuff like that. And okay, a bit of a gloating moment here, if you, if you all permit me. I, I, I hear other people now echoing the idea of threat model every story, so it, it, it must have resonated someplace with some people. But now, now I just can't shake the idea. I, I It's hard for me to see something that would not justify, if you work in an agile environment, not having threat modeling embedded as a foundational practice of your development team, of your product team, and perhaps offloading the more costly bits of the threat modeling activity The reporting, the risk management part, which is separate but works with, which is ranking the risks, figuring out the mitigations, but eliciting the threats. I can't in my head justify why that wouldn't be part of the Agile process anymore. So that the next Scrum Master shouldn't be asking the same question that Matt and I have been asking ourselves 14 years ago. How come there's nothing about security in this whole process?
45:35Matt Coles Ask a product manager.
45:38Izar Tarandach I even took that, what's it called? SAFE class.
45:43Matt Coles SAFE class is right?
45:46Chris Romeo Oh, dooms.
45:46Matt Coles Dooms.
45:47Chris Romeo I'm sorry.
45:48Matt Coles SAFE Agile.
45:50Izar Tarandach And I hope to see it there. And I saw mentions. I saw head nods. But still, it wasn't in it. It wasn't part of the process.
46:04Matt Coles And with the invention of scaled agile, which is, you know, multiple agile teams working together.
46:15Izar Tarandach There you go. You know, it's another one of those cases that, you know, if you save a bit every day, you're going to have, a bunch of money at the end of the day, at the end of the year, versus just take a week to play the lottery and hope that you get a huge windfall, right? So I, I don't know. That's when I— that's why when I see articles like this, my reaction in the beginning was, hey, this would work if it was X many years ago. Things have changed, like processes, delivery processes have changed. Even if you look into DevOps and all that supposedly good stuff, things have moved, right? So it's time for people to realize that, yeah, when you move things, perhaps you should take security with it.
47:08Matt Coles And, and benefit, benefit of the doubt, the author of this article does say at the very end, threat modeling should be foundation of secure application, secure application design. And it needs to be practical.
47:21Izar Tarandach Continuous, democratized, and embedded into agile workflows. I mean, you can't ask for more than that. I think that where we disagree is that I think that we are in a place where we can very well do these things, perhaps not ideally, right? But we have reached a point where you can do these things practically.
47:42Matt Coles And we know that, we know teams do. I mean, it's not, maybe not universal, maybe it's not all of the developers. Let's not even talk about vibe coders and companies that are built on that type of approach, but it happens.
47:57Chris Romeo Yeah. I think that's a good place for us to wrap up this episode at the conclusion of this article. And we found some common ground and we found some uncommon ground. I can't think of the right words.
48:11Matt Coles We did, we didn't, we didn't find a Security Boulevard article though. Yeah.
48:15Izar Tarandach No, no, that we did not find. That we did not.
48:17Chris Romeo So that appears to have been an AI hallucination, which is something that happens these days in the world of AI. But folks, once again, thank you for listening to the Security Table.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
