Capture the Flag or NOT?
With Chris Romeo, Izar Tarandach, and Matt Coles
There is an overemphasis on Capture The Flag in the security world. Instead, the industry should focus more on the 'builder' perspective to develop robust systems rather than the 'breaker' mindset typically associated with penetration testing and CTF competitions.
About this episode
There is an overemphasis on Capture The Flag in the security world. Instead, the industry should focus more on the 'builder' perspective to develop robust systems rather than the 'breaker' mindset typically associated with penetration testing and CTF competitions. In addition, we must shift the industry's reward and recognition structures to incentivize building secure-by-design systems.
A CTF is a type of cybersecurity competition where participants solve security-related challenges to find flags representing vulnerabilities or secrets within a system. A CTF and bug bounty are similar, as both test cybersecurity skills but have different goals and outcomes.
Red teaming is not just about penetration testing but also about testing the operations of the people who manage defenses.
Finally, the discussion ends with pondering the question of "winning" in cybersecurity and agreeing that providing a system free of defects and ensuring security assurance should be the ultimate goal.
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 41 min conversation
0:10Chris Romeo Hey folks. Welcome to another episode of the Security Table, we sometimes debate the philosophy behind the matrix. Was it the red pill? Was it the blue pill? But sometimes we talk about application security and other related things too. And so I feel like we're, we're still on this, this thread. Oh, who are we? Who are we? Who is anybody really? So I'm Chris Romeo, joined by my friends Matt Coles and Izar Tarandash. today, gonna continue pulling on that thread of what we affectionately referred to as reasonable application security. I think there's a, a connection with what we want to talk today, talk about today, but I think it's, it's also much bigger. And so the premise in front of us was
0:51Chris Romeo really an image that, that I saw somebody tweet and it was in regards to capture the flag. And so this individual who I should have been more prepared and ready to quote, give attribution to
1:05Matt Coles What else is new? I mean, come on,
1:07Chris Romeo So what else? We we're known at the security table for being very unprepared. Um, he, he, he basically just had this premise of capture. The flag is overrated. It's overhyped, it doesn't generate Doesn't help you generate CBEs. I thought it was a pretty interesting construct and it's gonna, it's gonna allow me to stand on a soapbox a little bit later and talk about what I think about red teaming and pen testing and the focus we give in the industry.
1:38Izar Tarandach Ooh,
1:39Chris Romeo let's, let's unpack the CTF question first. So, so you guys te tell me I'm not a CTF'er. Okay. I've never played a CTF, so I feel like I'm only qualified to share my opinions of it, not necessarily describe what it is. So, I mean, so let, let's work up. What is it? What is a CTF?
1:57Izar Tarandach Matt might you you go; you're good with the definitions.
2:02Chris Romeo Do you have a
2:03Matt Coles Great. great. the,
2:04Chris Romeo with CTF written on it and a definition?
2:08Matt Coles well, I, I, I, I do, I do have a flag with his on it. Uh,
2:14Izar Tarandach Consider it captured.
2:16Matt Coles So, so, so, yeah. Great. I feel I'm, I'm Mr. Dictionary over here. Great. Um, so CTF, right? Capture the flag. Uh, Competition for people to get together and, um, break stuff. Uh, well, not so much break stuff as find stuff, I guess is the important part, right? So we're talking about a structured competition, uh, with a defined goal in mind or set of goals, maybe some milestones along the way. Uh, and, and people, uh, get together and. And ultimately it's a competition, right? So, so there's, there's one objective, you know, a piece of data or, or a control of a system or whatever. Uh, and obviously we're talking, uh, we're talking se security here. There's other capture of flags. You know,
3:01Matt Coles if you're, if you're a gamer, you might be going for that literally a flag, uh, on the screen or you're doing paintball or something. Very similar concept. Uh, hopefully folks have that pretty clear in their mind. You're looking for something, you're trying to get it before the other team does, uh, capture it. Hold it. Destroy it. Do whatever you need to do with it. And, and, winter, winter, winter, chicken dinner,
3:25Chris Romeo I love a good chicken dinner. So Eza, what else? Like what else?
3:28Izar Tarandach So that that's.
3:29Chris Romeo provide?
3:29Izar Tarandach Yeah, so, so that's, that's some of the fun. Of course there, there's a whole bunch of, uh, cts that happen either like individually, you go and you, you have a specific set of problems that progress usually in, uh, difficulty. for each one of them, you get to that string. That is the flag. You apply to the interface, you get the next problem. And, um, We have seen some of those also applied in companies as an educational tool or in hackathons, internal hackathons. They, they're very, uh, popular for that, that kind of thing. And stepping away from the definition and, and going into the is this good for, is I think why we're here I to me, okay. First of all, uh, the competition part, I think
4:16Izar Tarandach that, uh, a lot of people that are in this professional are interested in the, the things that we deal with. Uh, are competitive by nature. So are good ways of, uh, pitting what I know against what you know, and coming out of something that's, uh, quantifiable out of it. Uh, if I have more flags than you, if I have more points than you, then uh, I win and you lose. And, uh, I'll enjoy your dinner and dinner, but, uh, what I like to question is the educational point, point of view, right? Cause. What, what would be to, in my, my head, what, what's the closest thing that, uh, you could parallel uh, uh, CTF? To me, it would be bounty, right? But the difference between both of them is that the CTF,
5:03Izar Tarandach by the definition can be exploited, and there is something at the end of the rainbow. So it's like the CTF is a, is a sure thing. The bug bounty is you go in and you apply the things that perhaps you learned even in the CTF and happen, may not happen. You don't know you're going blind. can it. Own experience. Do I know what I'm doing here? And that, that
5:29Matt Coles So or would you. Sorry, would you, so, uh, what you're calling others that CTF has a, has is a structure, is a structured activity that has definitive goals and milestones that are achievable. Right? That's, that's what you're ultimately saying here is a CTF. A CTF is a structure for deli with content that is known, that tests for skills as opposed to bug bo, which is content that is unknown. Testing for the, like what flags do we have? Not how do you get to the flags we know of?
6:02Izar Tarandach Actually, I am more simplistic than that. I'm saying going to the CTF. You gotta, you gotta guaranteed dopamine hit if you do A, B, C, D and you go into the bug bug, uh, bug bounty and you don't know what's gonna come out of it. Perhaps you find something and perhaps you won't. Right. So I, I, I, I do see the parallel between both of them. I don't think that they, working on one is, is a good ramp up to the other.
6:36Matt Coles Would you, would you characterize, would you say though that a bug bounty is a, is first and foremost the test of a system, whereas a CTF is a test of skills?
6:48Izar Tarandach Oh, that's a beautiful way of putting it. So you you mean that the, the target of the exercise is different.
6:57Matt Coles The outcome is different, right? Yes, you need skills to do a bug bounty, but you don't know what the target is. You don't know what that flag is when you do a bug bounty. So you have to have skill. You're testing skills, but you're really looking for the, the goal of running a bug bounty is not to know whether an attacker is good, but to know whether they find, find the flags, right. Find something, and, and again, those flags are. Unknown at the time. Whereas the CTF, all the flags and all the milestones on all the path through the system are known to the organizers. At the very least, you're testing the skills and, and potentially the collaboration and, and, um, and inner working, you know,
7:34Matt Coles interpersonal, uh, and, and technical operation of a, of an individual or a team to meet, to, to get to those objectives and milestones and, and the flag ultimately to the flags.
7:45Izar Tarandach Okay,
7:45Matt Coles So the goal of the, the goal of the exercise is different.
7:48Izar Tarandach so analogy time, it sounds to me like you're saying that the CTF is like a shooting range you have like very controlled situations, very controlled limits, and you are basically. Checking out if you know how to a weapon, how, how to shoot in a target,
8:08Matt Coles Mm-hmm.
8:08Izar Tarandach the bug body is closer to being out there and doing those things that people who do things with weapons do. You have all kinds of, uh, unknowns and, uh, the, the boundaries are not known. The, the, the terrain is not known. Who's shooting back at you is not known. So all that, and then of course, it extrapolates to bad people doing bad, bad things, which is actually the, the word of attackers attacking our applications. So three, three here of indirection. Yeah.
8:36Chris Romeo Let's give it a, let's take it a step further though, and talk about outcomes of each one.
8:41Matt Coles Mm-hmm.
8:41Chris Romeo about, when you're putting bug bounty against CTF, the, the outcome that's most, I think most people are aiming towards is enhancing their knowledge and the CTF, they to break stuff better so that they can apply it to breaking stuff in wherever else they break stuff. The bug bounty, the outcomes dollars in your pocket. So similar path, but. Bug Bounty. So I'm thinking about Matt Bug Bounty from the perspective of the person who's doing it, not the, organization who's
9:12Matt Coles there's, there's two. There's two aspects there that to consider. I think you're, yeah,
9:15Chris Romeo real CTF, like there's no real organizational value. I guess there could be organizational CTF, um, of, of enhancing the knowledge of the people that work in your company.
9:25Matt Coles well, I would add.
9:26Chris Romeo individual first.
9:28Matt Coles Oh, well, I, I, I would actually, uh, just you, I think you covered the bug bounty and the CTF from the goal of the participants pretty well, right? A test of skills. It's a test of skill. It is a test of skills, and there's money associated probably in both ends, right? There's, there's either fame and fortune from, and participating in winning a CTF and you gain, gain skills or you show off your skills, right? You either get bragging rights or you get a payout or you get a promotion, you get whatever. same thing for the bug bounty, right? Although that's more of a financial impact, or maybe it's swag or maybe it's a leaderboard or whatever the case may be. So for the participant, you get, you
10:03Matt Coles get tangible benefits from either of those activities and very similar tangible benefits. The bug bounty from a organizer's standpoint, the bug bounty is I get somebody else to do testing for me that I then re, uh, compensate them for. And so I learned better about my system and I learned then, uh, what, um, Uh, you know, I, I, I, I can improve that. A CTF, though, I think. is really from a, from an organizer's standpoint, if it's a corporate organization organizing it with their employees, they're testing their employees abilities. Maybe it's a team building exercise. So they get, you know, some interpersonal connection there. They get to collaborate, they get to test out processes. We can talk
10:47Matt Coles later about red team versus Blue team, cuz there's another area to go into. Uh, Or if it's a organization that does it like the us um, as a US CTF. Uh, I, I'll have to pull it cuz it was, that was at Defcon last year run by a good friend of mine. Uh, or, or, or, uh, at least mcd by a good friend of mine, uh, uh, who, who, um, this was, you know, bring teams together. They do CTF, um, they learn about new attack techniques, right? So it's a chance to show off new industry trends or learn about new attack behavior or, um, or, you know, show off, show off in a different way, right? So there are tangible benefits across the board. Not all of them are financial
11:32Chris Romeo So the original of Twitter post tweet got me to this, I just said Twitter post, I'm so embarrassed, uh, that got us to this conversation. Was an image with the header at the
11:45Izar Tarandach Facebook posted
11:47Chris Romeo right. Um, I put it on the Facebook. Um,
11:51Matt Coles You didn't ticket or talk it
11:53Chris Romeo I didn't, I don't understand that. I don't, I don't even want to go there. Matt, you, you almost, you almost open a can of worms suggesting that I would be on TikTok now. I'm not
12:01Matt Coles I,
12:01Chris Romeo of worms. That's,
12:02Matt Coles disclaimer, I'm not on TikTok either, so
12:05Chris Romeo that's a whole other, that's a whole other discussion, debate as far as where that data goes. But the
12:09Izar Tarandach The security table dance.
12:11Chris Romeo That this tweeter did was stop doing CTF security was not meant for competitions years of CTFs yet. No real world CBEs found. You want to a hack something? Don't use CF ti time use this. And then it's an analog modem with the, with the, so I don't know if it's a, if, if this is just purely a trolling that I just fell for, or, but it does bring this issue up like, are we too focused on CTF in the world of security and we're going outside AppSec. Now we're talking about security in general. Do we focus too much on CTF?
12:48Izar Tarandach Okay. Oh,
12:50Matt Coles Go Go ahead. Ard. You first.
12:52Izar Tarandach Last time that I participated in CTF was like, what, two years ago? And for one task, like the, the one that the points were like absurdly bigger than everything else. And it ended up like was one binary that had something like eight different layers of, of. Garbling and obscurity on top of it, and ended up going binary in one language, encapsulating another one, and so on and so forth until the flag, and know what it, it has absolutely nothing to do with security. It was more about reverse engineering and, and figuring out some very basic crypto, but I didn't care because it, it kept happening and the dopamine kept hitting and I kept solving a problem. and I could not give less of a care about
13:45Izar Tarandach am I having a security experience, am I learning more? Because it felt good.
13:51Chris Romeo Hmm.
13:51Izar Tarandach And I think, and my point is that, uh, we, we, we already have to fight so much when, when we are teaching, when we are pushing, when we are, uh, building these programs, we, we have to fight so much for people to get interested in it and for people to give you their time and, and their attention. and I think that that dopamine hit is, is a positive. If I can get somebody at least minimally interested in some aspect of security using that mechanism, I'll freaking use it.
14:20Chris Romeo but how practical, like you just described something that's never gonna happen in a modern web application architecture
14:26Izar Tarandach Oh, no, no, no, no. That, that, that was like that, that was just illustrate effect that I got so engrossed in into it. Right.
14:32Chris Romeo but
14:33Matt Coles And,
14:33Chris Romeo something that doesn't, that has no real world applicability though? Or is there some real world case case for this architecture
14:41Matt Coles You're think you're think you're thinking about it wrong. I think we're thinking about it wrong, and maybe that's, maybe that's, we're putting a lot, we talked about it. We, we just talked about it actually. CTF versus bug grati. Right. The goal of the exercise is different. We're not in a CTF. We're not looking for new vulnerabilities. We're not looking at necessarily inventing new attack techniques, although that probably happens in a CTF probably pretty frequently. But that's not the, maybe that's not the outcome that we should be asking ourselves about, right? If the, if the, if the statement is CTFs don't produce any vulnerabilities, therefore they are bad. Meaning their waste of time because
15:20Matt Coles they don't produce vulnerabilities. I would suggest that their as, that the, the goal, the expected outcome is the wrong outcome.
15:28Chris Romeo Hmm.
15:29Matt Coles it's not a waste of time if it's a team building exercise. It's not a waste of time. If it's a opportunity to get people interested in the field of cybersecurity, it's not a waste of time if it gets people thinking about, you know, new, new attack techniques or new memory defenses or new coding constructs or other things, you know, that are non-tangible benefits that come out of or are, you know, not, not directly related to the CTF benefits. Uh, and therefore it's not a waste of time. If you're looking to find defects, well, that's a bug bounty,
16:01Izar Tarandach Mm-hmm.
16:01Matt Coles right? If you want to do cvs, c cvs, that's a bug bounty. We don't know what the target is. When you construct a CTF, you plant a flag. You know, that's a, that's a target, that's a piece of data. And you say, oh, that data is protected by these things. And well, in order to get through that, somebody's gonna have to know how to do a memory injection and, and, and a heap, heap, heap overflow, and, uh, you know, route to, you know, return to lipps and all this other stuff. Well, They may find a way around those, those attacks and do something completely different, but that's not the outcome you're looking for. The outcome is basics or a particular path or a particular goal in.
16:40Chris Romeo I'm somebody who could care less about CVEs. And it's not just because I don't have any, let the record show. I just, I don't, I mean, I don't, I don't measure my worth in the security industry based on the number of mire, cve, e entries that exist in the database. But
16:56Matt Coles but we are not pen testers.
16:58Chris Romeo but I think when I was about ready to go. I'm gonna go there in that direction though, right? Like, I don't consider myself a breaker. Using the classic owas terminology, builder breaker, defender.
17:10Matt Coles Mm-hmm.
17:10Chris Romeo consider myself a breaker or really, I mean, we're really a defender. I, I consider myself a builder and as a builder I think about how can we spend our time better so that we can build better stuff that doesn't have the problems. Like if we can build something that doesn't have any of the issues, tech, they'll have nothing to break. There'll be nothing to find if we, if we, and, and so this is, I'm, I'm kind of, I'm about ready to unleash my, um, soapbox moment here because, and I've written about this before. I've talked about it publicly, and it's an unpopular opinion. Get ready to put up the unpopular opinion flag. I don't care. In our industry, we focus too much on breaking, we focus too
17:53Chris Romeo much on red teaming, too much on CTFs. Um, these guys are all writing the resignation letters to the security table right now, and so it's okay,
18:01Izar Tarandach Uh,
18:02Chris Romeo but we spend, we spend too much time. Like, and, and all you gotta do is go walk up. So find a, a kid who's, who's a kid, I shouldn't say that. A student, a young, a, a younger person who's in university, getting ready to go into cybersecurity and ask them, what's the number one thing you wanna do? And what do they all say? I
18:21Matt Coles Break stuff.
18:22Chris Romeo secure Java code. eliminate cross. No, they all say they wanna break things and so as an
18:29Izar Tarandach and move fest.
18:31Chris Romeo and move fast. That's fine. Move fast. That's me, That's my soapbox moment. Thank you. That is an
18:35Matt Coles is.
18:36Chris Romeo excellent graphic. CBEs. Who cares? I feel like I'm like, I'm the get off my lawn guy when it comes to, you know, that's, that's my new persona I'm taking on here. But, We spend too much What is this? Scary? What is it? Good Four? No, I mean, we spend too much time focused on it. Like, if we could take half of the effort we spend breaking and invested in building, wouldn't we be able to eliminate all the stuff? We're a lot of the stuff that we're breaking. Like imagine if a kid came out of a CS program on cyber and said, I wanna write more secure code. Ah, feel like it was a a, a great moment. I, I, I tell I'm on a rant. I'm in the, I'm
19:12Matt Coles They would, they would get laughed at out of that classroom so fast.
19:16Chris Romeo But
19:17Matt Coles But you know what,
19:18Chris Romeo what's wrong with Dennis Miller. I'm like, I don't want to go on a rant or anything, but I'm already there like,
19:22Matt Coles Yeah. so,
19:24Chris Romeo this rant. I
19:24Matt Coles so lemme let me throw the argument, let me throw the argument that I, that I think we always hear is well, In order to be a good defender, you have to understand what the attackers, you have to think like a hacker, right? You have to, you have to understand.
19:36Izar Tarandach no, Mad Mat. Mad Mat.
19:40Chris Romeo Exactly. Bad, bad. Matt is now being muted by the host of this podcast right now cuz I have the mute button and I'm muting him because Adam's sh Adam Showtek just called and said, did someone say, think like an attacker? Take them out. them off. The podcast. Eza is leaving. For people listening via audio.
19:58Izar Tarandach I, I'm, I'm, uh,
19:59Chris Romeo the set.
20:00Izar Tarandach no, I, I, I'm going to put Myst hat for a second.
20:03Chris Romeo Okay,
20:04Matt Coles There you go.
20:06Izar Tarandach did you just say think like an attacker?
20:12Matt Coles Wait, wait. So
20:13Chris Romeo like a hacker too. I heard that too.
20:16Matt Coles I'll, I'll just have to put money into the Jar Square jar for that
20:20Chris Romeo Now you're gonna
20:21Izar Tarandach for speed. Okay, continue Matt.
20:27Matt Coles Did you, did you fall? You fell off your soapbox apparently though,
20:31Chris Romeo Well, I'm still screaming at The fact that you said think like an attack. So you're where you, where you begin. I'm gonna re, I'll rehash just to prove I'm a, I'm an active listener that I did hear what you said before I started screaming
20:45Matt Coles did you really? Let's, let's see. This, let's, let's make sure you
20:48Chris Romeo said is the, the, the classic argument is that to be a good builder slash defender, you have to understand how to think like a beep. I couldn't say it, I can't bring myself to say
21:01Matt Coles And, and, and, and please note, please note for viewers at home. I, I don't feel that way. I'm just re re replying with what the counterargument is. Blah
21:11Izar Tarandach People see
21:13Chris Romeo Yes. He's
21:14Matt Coles People want good people on both sides. Say
21:16Chris Romeo need the pundit view. Definitely. So, but yeah. Let's, so let me, let's unpack that a little bit though. Like, is that true? I. Do, do actually, like, where do we land on that? Do we, do you believe like I, I mean, I think you
21:28Matt Coles of
21:29Chris Romeo you have to have, you do have to have a solid foundation to be a good builder on how things work.
21:36Matt Coles Yep.
21:36Izar Tarandach Yeah.
21:36Chris Romeo have to understand like if, if you're a developer, say you're an AppSec engineer and you tell me during the interview, Hey, I've never actually shaken it. I've never actually exploited cross-site scripting versus SQL injection, but I've studied them a lot like that. Is that like you do have to have that foundational knowledge? What I'm saying is like at the industry level, we're so hyper focused on breaking everything it, it's a detriment to. Focusing on more secure by
22:05Izar Tarandach Look, uh, I, I. Oh yeah, yeah, exactly. Yeah, exactly.
22:12Chris Romeo cws. unpack
22:13Matt Coles we have a source of, we have a, we have a source of, of, of ingra. We have a source of commonly known security knowledge that we can leverage. right? I don't need to know the technique that an attacker uses to exploit a se to exploit, uh, cross-site scripting vulnerability. If I understand how a cross-site scripting vulnerability gets introduced in code or is exposed in a network service, I can defend against it or I can build a avoid it in building. There's a lot of skills involved with how to do testing. Now, I will say I used to be a QA tester, uh, early on in my career, used to do security QA testing. So having that structured mind of being able to think about a system un identify attack
22:59Matt Coles points and put together a test plan is I think, important to, to know how that's going to happen. But being a pen tester to be the attacker. I cannot put myself in those shoes, right? I, I'm, that's not my mo right? And I don't think that goes for the, the two of you well.
23:20Izar Tarandach look, we, we, we have to be, we have to be honest about something. Uh, when was the last time that you saw a movie that shows that amazing point in time when somebody sits in front of a black screen and starts clicking, clicking, clicking, clicking, clicking, and says, I just wrote the library, the defense against success. It doesn't happen.
23:44Chris Romeo I'm gonna write
23:45Izar Tarandach What, what, what, what? What happens?
23:47Chris Romeo gonna watch it. Nobody.
23:48Matt Coles Champ, G P T Champion, G Champion G P T a script for you and you can
23:52Izar Tarandach what
23:52Matt Coles act it out.
23:53Izar Tarandach scanning the power plant. What happens is, uh, the guy that's played tour doing, I don't know what, with uh, uh, some, some, anyway, my point is pen testing. sexy. Defending is not kids coming up the ladder. Many of them, they are going for the fame, for building their nicknames and stuff like that. Nobody build their nick, sorry. Most people don't build their their name by building stuff for defending stuff. They do it by breaking stuff. And we have some great, great, great examples of people who started breaking and came to the other side of the table and started defending it and done it wonderfully and moved the the state of the art forward. Right. even if we look at the black hat, it's sort
24:42Izar Tarandach of. New Wish that Black hat has a defender track, right? I don't know how how many years, but it's not recent.
24:48Chris Romeo Yeah, true.
24:49Izar Tarandach It started as, as, as a full on black hat thing. And I think that of course, that there's a reason for that. Fantastic. breaking things. It's sexy defending. It's not, we, we are the gigs of the gigs
25:06Chris Romeo Yeah, that's that's true. And, and let the record show, I did just have chat. G p t write a short movie script write where a developer writes a secure library to save the day. So it's in, we're now accepting investors if you wanna be a part of this project. Um, I'm gonna, I'm thinking maybe Canu, Canna Smith
25:24Izar Tarandach I don't know. I, I don't know. I see a new career for me here. Like, look at this profile,
25:31Matt Coles need, you need to get, you need to get gal Gado or somebody you need. Have some, get a little diversity in there. You know,
25:36Chris Romeo I mean it's, you know, this is, this is, this is a, uh, this is my next project. I'm gonna work on this. This is, it's gonna be a short movie and no one's gonna ever watch it, but that's okay. So,
25:45Matt Coles you'll be surprised. You'll be surprised they'll be probably the 800 views by the time you.
25:49Izar Tarandach no, back, back, back to our thing. Okay, so we agree that fantastic sexier than, than defending. Now what do we do with that?
25:58Matt Coles Well, more important and actually a reason why pen testing is more, is more sexy than, than defending what happens when you pen test. There's bashing on the keys. There's stuff happening on screen, right? There's something to show and there's something to hear, and there's the, the sound coming from, from the people who are doing it. On the flip side, you have the defenders who are running around like crazy, you know, crazy people with their head cuts, cuts off, cut off. Oh my God, this guy is falling. They breached our network. What are we gonna do? Or step one of the policy says, do x. Step two says, and, and there's not much to see. Right? uh, you know, technology demos are really hard when
26:42Matt Coles there's nothing. Um, sorry, some sort of bug is flying around. Um, technology demos are hard when there's nothing to show. Right. A couple lines on screen running
26:51Chris Romeo Yeah.
26:52Izar Tarandach Wait, wait, wait. You sound like someone who has never sat in front of the logs coming from a firewall and going, yes, yes. I see your.
27:02Chris Romeo That was like a Gollum thing going my
27:05Matt Coles you were, if you were, if you were Uper a honey Pott, would you sit there and go, oh, he didn't do, oh, he did that. Oh, he, he's gonna go there. He's gonna go there.
27:14Izar Tarandach You, you
27:14Matt Coles But would you watch that? What would that?
27:17Chris Romeo I came up in, I came up in the world of computers at it that was actually happening. Live on we're like, hold, what did he just do? Wow. He or she really just took off, took down our entire system here. That was pretty cool. You know, that was the defender. But I mean, Eza to your point, like what do we do? And I mean, I think at the end of the day, unfortunately there's not a lot we can do because is the motivation, it's the extrinsic motivation that drives people. To want to break stuff and to get the fame and the glory that comes along with doing it. I think as an industry, we should be investing more on the defender of the builder side, and that's what, you know, you guys are the same as
27:58Chris Romeo me. We've, we've, we've dedicated our lives in our careers over the last fif 10 or 15 years to doing this. Do I think we're gonna move the needle? A lot. No, but I think if we can do something to influence the people inside of big companies that are, that are, that are looking at the bottom line and then return on investment for what they're doing in security, I think over time people and people are migrating more towards secure building over the last 10 years than pen
28:29Izar Tarandach Okay. Off the top of my head, Chris, what if your next project, instead of the movie was we reversed the model of the CTF and now rather than going and getting the flag, you have to protect the flag. We get engines to do automatic attacks and you get to see the attack happening, and you get to stop that thing.
28:50Chris Romeo that does that. People, people have Yeah. There's
28:53Izar Tarandach I haven't
28:53Matt Coles Yeah.
28:54Chris Romeo There's a company that's doing blue team where I think
28:58Izar Tarandach oh, and I missed the
28:59Chris Romeo Yeah,
29:00Matt Coles and remember N E C C D C.
29:02Izar Tarandach Yeah, but that's live like that. You have to people attacking you. But it's a, it's a red team, blue team and they focus on the blue team is the, the, they don't, they don't give points for the red team break breaking the blue team spirit. They they give points for the blue team. Having a system le is live and running at the end right?
29:25Chris Romeo and it's, it much just to be very clear with the terms, right? Those environments are. breaker versus defender, not so much breaker versus builder or or breaker versus builder and defender. When I think red versus blue, it's, it's the defender that's representing, that's, that's protecting the system more at the network level and the service level versus writing better and secure stuff from, from the start.
29:53Matt Coles Now can we, can we just take a quick, uh, sidetrack here? We talked, we started with CTF versus Bug bounty. We didn't talk about red team exercises.
30:02Izar Tarandach Good
30:03Matt Coles there is a very, very similar thing, right? Red team, red teaming, well, I'll use my definition whether it's, uh, you'll, you guys can tell me if I have it right here. A red team is a test of, of an operation, a test of an operational system where you're testing both its defenses as well as the operations of the people who are managing it. The defenders function,
30:24Chris Romeo Agree.
30:25Matt Coles right? It is not testing.
30:27Izar Tarandach mm-hmm.
30:28Matt Coles not print testing per se, and, and, and people often confuse them. I think that they call pen testing and red teaming or bug bounty and red teaming to be the same thing, but they're not. Their goals are, are their, at their techniques are the same or, or very similar, but the goals are different.
30:45Chris Romeo Yeah, I think in the industry there's, there's a lot of people have, have used those as synonyms now, cuz you know, if you're doing a pen test with a physical component, technically you're red teaming, right? Cuz you're, you're going beyond the technology, you're testing the processes and the people at the security station, at the gate, at the, the door.
31:05Matt Coles May maybe if you're, if you're, if you're test well, if you're, yeah, I guess if you're, if you're doing physical penetration testing and you're gonna break in, you're trying to break in without being detected, I guess technically that is red teaming. Although your goal is to find the vulnerability, not necessarily to make sure that you don't get caught.
31:22Chris Romeo Yeah.
31:23Matt Coles Right.
31:24Chris Romeo I mean, some red teaming engagements, pen testing engagements. The goal is not to get caught, is to try to get in, violate the physical security constraints, violate some logical security controls, walk away with the computer out of the data center if you can do it and sell
31:42Matt Coles Right. Make best friends with the, make best friends with the security guard while you're at it, you know
31:46Chris Romeo an attack of, of the, the policies, like you said, your definition was, it was beyond the, the technical controls. It was attacking the policy and the people behind the of the system. is a lot of time what's happening there. So, I don't know. I mean, where do we, where do we go with this? Like, should we start a movement? Should we start a, a new, a new manifesto? You know,
32:09Matt Coles Well, how would you, so let's start with how would you. How would you incentivize this? So I guess if you look at bug bounty and you look at a CTF, right? You can incentivize behavior. Behavior is incentivized because it's sexy. There's stuff to see, there's stuff to hear. Um, there's, there's financial payout, there's a leader board, whatever. How would you incentivize, not the defender, but the builder side, how do you incentivize that, let's say from a competition standpoint? how? How would you even approach that?
32:42Chris Romeo I mean, off the cuff, I think outside the constraints of a single. Competition. That's really what security champions exist for inside of companies is you, your security champions program is to try to provide a reward and recognition structure for people doing the right thing a builder perspective. So I don't know, do we have the security champion Olympics
33:07Izar Tarandach No, but it isn't, but it isn't the, the, the reward is in the doing itself. I mean once, once, okay. So Security City, we can look at it and say it's a hobby thing. It's, it's a, you are interested in it thing. A pen test. It's a professional thing. It's you paying somebody to go and poke at your stuff.
33:25Chris Romeo Mm-hmm.
33:27Izar Tarandach the incentives there are are pretty different now from the builder point, point of view, incentive is the thing itself. If the pen test doesn't find anything, if the bug bounty doesn't find anything, I won. And I win by doing the right things.
33:43Matt Coles Mm-hmm.
33:43Izar Tarandach we are going back to the discussion that we always have to, how do we incentivize developers to do a bit more of what we want them to do and a bit less of what they are already doing. So I think that we, we are extrapolating now to, to a whole, system of incentives that feeds off each other.
34:02Matt Coles Okay. also have to think that this is a long, this is a long game. Discussion. Right, right. You to start with No, no. Yes. But I mean, from a, from a system development standpoint and a and a developer's standpoint, they have to be com. They, in order to be, um, in order to get the reward, they have to delay gratification. I'm gonna build this thing, and then at some point later, I'll know if I have won. And by the way, do you win at release? Do you win at that first pen test and they don't find anything? Do you win 10 years from now when your, your system, um, escapes unscathed with no CBEs,
34:40Chris Romeo But we're trying to get bigger. We're trying to get bigger
34:43Izar Tarandach You don't win.
34:44Chris Romeo if you want to, if you
34:45Matt Coles but you do,
34:45Chris Romeo look
34:46Matt Coles you just said.
34:47Chris Romeo look at how the pen test look. Look at how pen testing and breaking is so top of mind in our industry. It's a thing everybody wants to do. Like that's so much bigger than we're talking about tactical, incentivizing developers on the ground. And maybe that's the foundational layer that we're missing. But I wanna get to the point where breaking and building are at the same level from an industry perspective.
35:11Izar Tarandach But it won't. It won't, it won't. Because when you pen test something, of, uh, delayed gratification, if you don't have any findings and you got a good enough set of pen testers, and that's a whole different discussion, how you choose your pen tester. Then from a certain point of view, you say, at this point in time, apparently I don't have any glaring issues. Right. On the other hand, if you're defending they didn't find anything, You haven't quite proven anything to yourself. You have just proven that a group of people have not found anything at this time, so you, you are trying different set of assumptions.
36:00Matt Coles Well, the problem is you're pro, you're trying to prove it negative, right? You're trying to, you're trying to prove you're free of defects,
36:07Chris Romeo And
36:07Matt Coles which is
36:08Chris Romeo security assurance, right? That's where I grew up in security, was going through the assurance process and trying to prove. And it got into the levels that I didn't even understand of formal methods and things. I never that worked. I can pretend like I know what it, what it actually is, but,
36:27Matt Coles A lot of math. Yeah. It's just
36:29Chris Romeo but at the end of the day, it doesn't work for an operational production system because you takes you six months to prove the version you released six months ago is secure. And by that time we've generated, you know, 200 releases a day. And so it just, it doesn't, it doesn't add up.
36:45Matt Coles I, I, I Good. Yeah.
36:46Izar Tarandach go there. I'll go there. You, you mentioned formal methods. Absent of formal methods, the only thing that you can actually say is my version is secure. To the best of my knowledge right now, that anything. In 15 minutes, in, in 15 years.
36:59Chris Romeo True.
36:59Izar Tarandach So to quote, the best movie ever made poster here in the corner, only way to win the game is to not play it.
37:06Chris Romeo Yeah.
37:07Matt Coles Well, so. We can't not play it here, let me let, let me, let me, let me flip this a little bit and maybe, you know, let's think a little bit out of the box. Throw the security aspect out the window for a moment. What makes developers want to be developers? Because developers are builders, not security people, they're builders.
37:33Izar Tarandach Yeah.
37:34Matt Coles makes them want to be engineers? What makes them want to be developers and programmers solving problems? And you invent and you incentivize it by giving them problems to solve.
37:44Izar Tarandach Ah, solving the problems that they want to solve. Not all problems.
37:49Chris Romeo he's bringing him, he's bringing us full circle. I see where he is going. can see two steps ahead. I can see where he is going. He's bringing us back around to the CTF and saying, CTF is solving problems. We just need it to be something that's tailored towards developers. so that they can, they can solve their kinds of problems versus
38:08Izar Tarandach But, but then Chris, goes back to the experience that you have with the, the training stuff and all that, right? When we started gamifying all that stuff and giving people snippets of code for them to go and, and do the right thing because we just. Show them not it. We, we know how that, that it works better than not having it, but that it didn't really get to people's imagination.
38:29Chris Romeo Yeah. It industry. The,
38:31Matt Coles there's a, well, there's a missing.
38:33Chris Romeo changed.
38:33Matt Coles a missing, sorry. There's a missing link there. Right? When you do that, you're just sort of, you're just, it's just the skills test, like within the scope of that training exercise for the developer. It's not the real world application, right? So how do you take that snippet, or how do you take that, that exercise, not the snippet itself, but the exercise and bring it back to the defense of the thing the developer just built, because that's where you'll get the full value from it.
39:00Chris Romeo GitHub needs to add locks. and every time somebody does something somebody can nominate them for a lock when you're doing your code review and then, the security and, uh, GitHub, that is pat hub, pen pat and petting trademark. Um, not really, but like, but I mean, imagine that type of a external of validation like GitHub Stars is a great validation for like, when you look at an open source project, if it has. Two GitHub stars, you're like, eh, maybe I'll find a different prep package to use. This doesn't look great. Imagine on your GitHub profile though, if you could have that where other people had recognized a, a positive security pattern and gamified it for you.
39:41Izar Tarandach collision, collusion,
39:47Matt Coles There's a, maybe there's an AI opportunity there, but, uh,
39:49Chris Romeo modeling guy that comes back and starts pointing out the flaws
39:53Izar Tarandach What? What do we do here? We have, we have three of those here.
39:56Chris Romeo I know. That's the challenge. That's the challenge. I thought I had a good idea, but All right, we're outta time on the security table for today. Um,
40:03Matt Coles We are making
40:04Chris Romeo like we solved. I think we, we made some progress in the right direction.
40:08Izar Tarandach Wait, we were trying to solve anything
40:09Chris Romeo Oh. I'm always trying to solve for
40:11Izar Tarandach Nobody told me.
40:12Chris Romeo Oh, I should have told you that upfront. But, uh, I think,
40:15Matt Coles you didn't capture the flag.
40:16Chris Romeo It was a good, it was this good and spirited discussion. I mean, I think that's the highest easer and my blood pressure has been during an episode a result of Matt's, uh, describing, think like a beep. Um, I can't even say it. I can't even bring myself to say it, but we'll continue our spirited debate because Matt tried it at home and Za is representing now a Star Wars bomb wearing So let the record show he
40:47Matt Coles Wait, wait,
40:48Chris Romeo Darth
40:48Matt Coles that Star Wars or Space Balls?
40:51Chris Romeo Oh, that's true. It'd be bigger. Nah, it'd be bigger space. He
40:55Izar Tarandach Prepare for low degrees speed. So everybody have a, a ludicrous Memorial day and uh, enjoy your weekend
41:03Chris Romeo Yeah, enjoy your weekend. Have a great time. Thanks
41:06Izar Tarandach I'll see you guys on the next one.
41:07Chris Romeo Yeah, thanks for listening to us on the security table. Thanks.
Transcript supplied by Buzzsprout. It may contain transcription errors.
View the original transcript on BuzzsproutOriginally published as The Security Table. Part of the AI Security Table archive.
