The End of Bug Bounty As We Know It
With Chris Romeo, Izar Tarandach, and Matt Coles
If AI can find and validate vulnerabilities faster than people, why would a company keep paying outsiders to report them? Chris Romeo, Izar Tarandach, and Matt Coles start with Linus Torvalds' changing assessment of AI-generated Linux kernel reports, then examine what useful automation does to the bug bounty economy.
Part of: AI in security testing and bug bounty
Episode chapters · 12 chapters
- Intro and book-writing detoursAudio
- Linus Torvalds and AI-generated reportsAudio
- When AI bug reports become usefulAudio
- Shorter experimentation loopsAudio
- Guardrails for defenders and attackersAudio
- Model costs and provider monitoringAudio
- Does AI spell the end of bug bounty?Audio
- Validating the flood of reportsAudio
- Bugcrowd submission volumeAudio
- Linux kernel review still mattersAudio
- AI books and simulated personalitiesAudio
- Wrap-upAudio
About this episode
If AI can find and validate vulnerabilities faster than people, why would a company keep paying outsiders to report them? Chris Romeo, Izar Tarandach, and Matt Coles start with Linus Torvalds' changing assessment of AI-generated Linux kernel reports, then examine what useful automation does to the bug bounty economy. They debate disclosure incentives, model restrictions that may constrain defenders more than attackers, and the cost of separating real findings from a flood of submissions. Bugcrowd's reported increase in volume brings the pressure on triage into focus. The conversation asks whether AI validation becomes mandatory, whether internal agents displace public programs, and what remains valuable about human research when both sides can automate the hunt.
Mentioned in this Episode:
➜ Bugcrowd
➜ The Linux Kernel Archives
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 42 min conversation
0:10Chris Romeo Hey folks, welcome to another episode of the Security Table. This is Chris Romeo, joined as always by Matt Coles, Izar Terrindasch. These are big AI thinkers, people. They, they think about AI as much as an LLM does.
0:27Izar Tarandach Which is not at all.
0:29Chris Romeo Do androids dream of electric sheep?
0:34Izar Tarandach Hey, that would be a nice book. That's a huge book.
0:37Chris Romeo I should write a book called that. Right?
0:39Izar Tarandach And eventually a movie.
0:41Matt Coles I think you could make a good movie out of that.
0:42Chris Romeo And then another bad movie that's set 50 years in the future after the horrible video game.
0:47Izar Tarandach And a couple of horrible video games. I think you got something there, Chris.
0:50Chris Romeo I got a franchise.
0:52Izar Tarandach Wait, where's my Claude? Wait.
0:54Chris Romeo Yeah, I'm going to have Claude write the script for me real quick. Or should I write the book first? I don't know. Will Claude write a full, a full book at this point?
1:05Matt Coles Yes.
1:05Chris Romeo I tell it to just write me a fictional book that's 300 pages.
1:08Izar Tarandach Last week, last week I, okay, I have to confess, I am a subscriber to Kindle Unlimited because I have this strange compulsion to read books and you read books these days. Yeah. And you get like, you get. Millions of, of, uh, of books by, uh, like free. And I fell into, uh, into a, uh, a well of alternate history military fiction. So think the Nimitz, the, the, the, that Nimitz movie where it goes back to World War I, uh, World War II, but variations of the theme. And I fell into this one where, uh, an aircraft carrier goes back to, uh, Mexico in 1519. Oh, and it starts nice, but you're reading the thing and then you start seeing those telltales, those, it's not this, it's that, those, you know, and I was getting this uncanny valley feeling of a person did not write this book.
2:20Chris Romeo So I'm working on my, my first number one hit right now.
2:24Matt Coles How do you, how do you churn out a million books at a time? You, you press a button?
2:28Izar Tarandach So basically, basically, but now I keep thinking to myself, remember those, those guys that do those, uh, uh, detective stories, they're like the, the B— Richard Sabin or whatever.
2:41Matt Coles The Hardy Boys?
2:42Izar Tarandach Yeah, yeah, yeah, no, no, no, the other one, the, the, the, the, what we used to call What you used to call airport books. Like they already put out like 6 a year or something.
2:53Matt Coles Oh, you mean, uh, uh, yeah, uh, trash, trash fiction.
2:57Izar Tarandach Yeah, yeah, yeah, yeah, yeah. So imagine how much is going to start coming out using AI.
3:03Chris Romeo That's what it already is. My book is cooking.
3:05Matt Coles It'll be slop fiction.
3:07Izar Tarandach Slop fiction! You heard the term first here.
3:11Matt Coles As opposed to Pulp Fiction. Slop fiction.
3:14Izar Tarandach Oh, now that's a parody I would pay to see.
3:17Chris Romeo Oh man, Claude just ripped off a book I'm reading for my book.
3:22Izar Tarandach I'm not kidding.
3:24Matt Coles All you have to do is mod— you have to modify a little bit and then you can copyright it.
3:31Chris Romeo Well, the name of my first book is The Long Quiet, so get ready. That's it.
3:35Matt Coles Long Quiet.
3:36Chris Romeo It's working on the, the chapters.
3:39Matt Coles Are, are you, are you making one that's will be movie ready and you can do like Uh, I should have told it, I should have put that in the prompt.
3:46Chris Romeo Dang it. I didn't do that. I, I just told it to write me a book and it's creating me a 90,000-word fictional sci-fi book and it's like, do you have a premise? I'm like, no, I got no premise. I got nothing. So just make it.
3:58Izar Tarandach If it has any sense, the names of the cap, uh, chapters will be Short Silence 1, Short Silence 2, Short Silence 3, and together they make Alone Quiet.
4:10Matt Coles Just silence at the end.
4:11Chris Romeo I like that. Yeah. I might hire you.
4:14Izar Tarandach And the rest— No, no, no.
4:16Matt Coles And the final line of the book is going to be, "And the rest was silence." You need to have the premise, a tech mogul goes and buys a farm in the woods. What can go wrong?
4:33Chris Romeo Thanks for calling me a tech mogul.
4:36Izar Tarandach Isn't that how The Last of Us starts?
4:39Matt Coles It's how every Neal Stephenson book starts.
4:42Izar Tarandach True. Very true., but he has to be digging for something in the farm. Like the farm is—
4:48Chris Romeo I gotta work in the A Quiet Place as well.
4:50Matt Coles Mm-hmm.
4:51Izar Tarandach Oh yeah.
4:51Chris Romeo Just cuz that movie was brilliant. You know, I finally, I finally, I was like an ignorer of A Quiet Place for years and years. I'm like, it can't be that good. And then I'm on an airplane a year or two ago, I'm like, fine, I'm gonna watch it. I'm like, wow, that was actually really good.
5:08Matt Coles And then you need to work in The Last of Us, of course, because that's like the thing.
5:12Izar Tarandach So that's the one where the aliens came out for sound.
5:15Chris Romeo Yeah. So that, so there's no sound in the whole thing. There's no dialogue in the whole movie.
5:22Izar Tarandach I could use one of those aliens when I was in the dorms at university.
5:27Chris Romeo It just rips through and takes your roommate away. Yeah. Except you gotta be able to control it. That's the only problem.
5:35Izar Tarandach Ah, life goals.
5:36Chris Romeo Life goals.
5:36Izar Tarandach I, I'll just—
5:37Chris Romeo all right. Well, yeah, we did actually have something to talk about before I derailed this. We did, we did, we did. My New York future New York Times bestseller.
5:45Matt Coles There you go.
5:46Chris Romeo What was it called? The Long Quiet.
5:48Matt Coles The Long Quiet.
5:49Chris Romeo For the, for the media tour, I have to memorize some of these pertinent facts about the name of the book and what it's about.
5:56Matt Coles Well, you just have the media talk to your agent, right?
5:58Chris Romeo Because, ooh, how far away are we from the, I have the little AirPod in my ear and the agent is telling me what to say and I'm just, it's available now, isn't it?
6:09Izar Tarandach Oh, that's the delay.
6:10Matt Coles Okay.
6:11Chris Romeo Yeah, I'd be afraid to just start talking like the AI was a teleprompter in my ear. So there's this guy, this guy, this guy, Linus, Linus, Linus is what he goes by. What? Linus Godavald.
6:27Matt Coles That's how we're going to get viewers, by mangling the name of the famous creator.
6:32Chris Romeo Nerds everywhere, picket us and hold up signs.
6:36Matt Coles Come on people, we're security.
6:37Chris Romeo Say his name. Yeah.
6:40Izar Tarandach It's sliders as in GIF.
6:44Chris Romeo Yes, correct. At least you said that right. All right.
6:48Matt Coles So it's like GIF.
6:51Izar Tarandach No, that's Linux.
6:52Chris Romeo Let's just start this again. Look in the file format, people. Analyze the first block of the file and that will tell you what it's called.
6:59Izar Tarandach Yes. Whatever's written on the terminal when you write the file.
7:02Chris Romeo There is a GIF in the file format. The, the block in the file header. I promise. At least there used to be. They probably took it out now. Um, okay, so it's a .gif extension. Our friend Linus came out about 6 months ago with some harsh words for AI, saying it was basically 90% of AI. This is October 2024. Uh, the, the Register referred to him as the Linux Kingpin, which is kind of a cool, it'd be a cool like X handle or whatever. The Linux Kingpin.
7:37Matt Coles He kind of is. I mean, he created the thing.
7:41Chris Romeo I mean, that has like a criminal kind of connotation though. And I think Kingpin, I think it's better than the Emperor Penguin. True.
7:51Izar Tarandach But he's been called that too.
7:53Chris Romeo But in October 2024, he came out and said 90% of AI is marketing hype. Which I think in October 2024, he probably was pretty close to right. 'Cause there was a lot of fearmongering and whatnot about how AI was gonna make your job go away. And I think there was, I don't feel like people are as afraid of that now as they were. Matt's kind of shaking his head like he might be afraid that AI's gonna replace him. Now I'm concerned.
8:27Matt Coles Have you seen, have you seen some of the Frontier model work? Hmm. Oh, it just actually, I mean, it's not really, I guess it's not really AI replacing us. It's companies replacing us with AI.
8:41Izar Tarandach Yeah.
8:42Chris Romeo Okay.
8:43Izar Tarandach I see what you're saying there. It's HR replacing us with AI.
8:49Matt Coles There you go.
8:50Chris Romeo I got an idea. Why don't we take it back? Why don't we replace HR with AI?
8:55Izar Tarandach Don't tell them the plan.
9:00Chris Romeo HR.
9:01Izar Tarandach Ugh. Nothing to see here, HR people. Nothing to see here.
9:04Chris Romeo There's no one from HR listening to this.
9:06Matt Coles Yeah. No. There's no HR people. Why would people listen to us?
9:10Chris Romeo Why would anybody listen to us? That's a more important question.
9:14Matt Coles How many agents do we have listening to us? Because that would be interesting.
9:18Chris Romeo Hopefully a lot.
9:18Matt Coles Yeah.
9:19Izar Tarandach So I'm going to tell you guys, I went back into the comments on YouTube and we actually have a fan from Brazil.
9:28Matt Coles Ooh, how do you know?
9:30Izar Tarandach Because he said, I'm here from Brazil. This is how you do right attribution, people.
9:39Chris Romeo This is how you build a community too, by interacting with each commenter one by one, because there's only a few. Okay, so why is this so such a big deal. Well, um, Linus is a— or Linus, depending on what—
9:57Matt Coles he's kind of important.
9:59Chris Romeo Yeah, I mean, he's the creator of the Linux kernel and still the chief maintainer of the Linux kernel. So he's, he's not just somebody with an open source library that 5 people have downloaded.
10:11Izar Tarandach That's me, that's me, that's me.
10:15Chris Romeo This is, this is somebody who's, uh, kind of a big deal in the, the world of open source. Um, and so he's quoted also in this article as saying he was willing to absolutely put my foot down as a top-level maintainer. Linux is not one of those anti-AI projects, and if someone has issues with that, they can do the open-source thing and fork it. So I don't think he actually meant that they could create an alternate copy of the Linux kernel. I think he might've had some other.
10:47Izar Tarandach I think he did.
10:49Chris Romeo But I think it also had some kind of— I don't know.
10:52Izar Tarandach I don't know. I think that for him to tell somebody to fork it in the first sense is much more offensive than the second sense. Because in the first sense, it's basically saying, go and do a better job than I do.
11:06Chris Romeo Yeah, that kind of hurts your feelings.
11:07Izar Tarandach The other one is sort of like, yeah.
11:11Chris Romeo That's true. Or just walk away. And then he described AI as a tool, just like other tools we use. It's clearly useful one. It may not have been that clearly even just a year ago, but it's no longer in question today. Anybody who has doubts, that clearly hasn't actually used it.
11:28Matt Coles And then he goes ahead and hacks on the maintainers, a little dig at the maintainers probably, because it's not like natural intelligence is always that great either.
11:39Izar Tarandach Yeah, exactly. Exactly. Gotta agree with the man there.
11:44Matt Coles In other words, we all make mistakes and the AI makes mistakes.
11:48Izar Tarandach Or we all write bad code.
11:50Matt Coles Which we do because that's, I mean, that's why there's a maintainers group and that's why there's bug fixes and I would say how many kernel flaws have been found recently.
12:00Chris Romeo The other thing that he kind of, when he kind of to sum or kind of wrap up what he said, and I quote, something happened a month ago and the world switched. Now we have real reports. All open source projects have real reports that are made with AI, but they're good and they're real. So what he's referring to, correct me if I'm wrong, but bug reports and whatnot that are being either found, you know, bugs are being found and the report is being drafted, written by AI and then submitted. It sounds like he has identified a, a switch. Where it was garbage in, garbage out a few months ago, and all of a sudden those bug reports are of value now. Is this the rise of Fable/Mythos class models that's doing this, or is this something else?
12:54Matt Coles It's probably related to that. I mean, we've seen this before with curl, right? Curl famously, I think, was getting inundated with garbage reports, right? AI slop. And they, I think they eventually shut down their bug bounty program because they were getting so many crappy reports. And then, uh, and then one day I remember seeing that Curl said they actually had a valid submission, at least hopefully at least one, uh, that, you know, that came from, from an AI-based report. So the models have evolved and they're becoming more accurate now, especially like Mythos and some of the newer ones that are able to do not only advanced reasoning, but actually do exploit generation. It's not only can it report a bug, but it can actually have the exploit code to demonstrate that the bug is real.
13:54Chris Romeo Have you guys been tracking these new Chinese models?
13:57Matt Coles Like QIME, that just dropped?
14:00Chris Romeo Apparently they don't have the same guardrails. Izar loves the word guardrails. That's why I use it so often in regards to LLMs. But they, whereas I saw somebody post something on X that had just gotten access to QIME and said, while OpenAI and Anthropic, he was getting cyber, he was hitting cyber guardrails is what they, he referred to it as.
14:26Matt Coles Yep.
14:26Chris Romeo In KIMI, it was wide open.
14:30Izar Tarandach It has to be said that OpenAI and Anthropic did say that their guardrails sensitivity was dialed up quite some numbers for now. Yeah.
14:42Chris Romeo Yeah.
14:42Matt Coles Because my understanding, I mean, Fable, if I understand correctly, I don't know all the details, but if Fable 5 is basically mytho, is basically like Mythos in terms of how it operates. Um, and it falls back to Opus and things when it triggers, uh, uh, one of the guardrails because otherwise it could do things like create bioweapons.
15:04Izar Tarandach I mean, but again, you know, it's, it's, it's, it's funny this whole thing cuz it's not like it's creating bioweapons, right? It's, it's, it's in there already.
15:17Chris Romeo It's giving you the ability, the, the knowledge if you don't know how to do it, to potentially do it.
15:24Izar Tarandach Right. So I was, I was listening to this podcast the other day and it was two very, very smart guys discussing how they are building the science lab of the future. So they're training their own models with what's it called? RL. I forget what the R stands for. And they were actually hooking up the models to a PCI bus that's connected to a lot of lab equipment. And they actually had the models create whole chains of experiments that they could run way faster than people and learn from the experiments, from the results of the experiments, to the point that they could predict the result of an experiment. So that reduced, uh, I think that's the first place that they were going with was, uh, uh, material science. That reduced the loop of creating a new material from 6 months to 4 days.
16:26Chris Romeo Wow.
16:29Izar Tarandach Now in here we, we are talking about experiments. We are talking about real instruments in a lab doing real science, feeding back into the model and the model actually learning from that. So apart from the sheer coolness of the whole thing, it's freaking scary because that's the kind of thing that creates a bioweapon, right? Not some kid going into Claude.ai and typing, hey, how do I make anthrax for my school project?
17:00Chris Romeo It just made me think about the— remember the Anarchist Cookbook? I'm sure you guys had a copy.
17:05Izar Tarandach Ah, I can't remember it because I didn't read it because I was never exposed to it on the internet. But let's say that I did.
17:11Chris Romeo That's, that's a really good admission of non-guilt. That was good.
17:15Matt Coles That's, that's exactly what somebody who read the Anarchist Cookbook would say.
17:19Chris Romeo Exactly.
17:19Izar Tarandach Let, let's go back to that article that we did. Yeah.
17:21Chris Romeo But think about how many copies of that existed.
17:24Matt Coles At least 10.
17:25Chris Romeo And how many people made Molotov cocktails? 'Cause I believe it did tell you in there how to make a Molotov cocktail. Yeah.
17:31Matt Coles I think it did.
17:32Izar Tarandach It's been rumored.
17:32Chris Romeo Yes.
17:33Izar Tarandach Yeah.
17:34Chris Romeo But, but how many people made Molotov cocktails? Some, some, some percentage, but it's not like 50% of people that read the Anarchist Cookbook And because a Molotov cocktail is actually pretty easy to make. It doesn't take a lot of—
17:45Izar Tarandach let me stop you there for a second.
17:46Chris Romeo A lot of science, please.
17:47Izar Tarandach Let's go back to the beginning. We were talking about The Last of Us. We were talking about, uh, I, I don't think that we did The Walking Dead, but we, we were around that area.
17:55Chris Romeo Love the early seasons of The Walking Dead.
17:57Izar Tarandach In those movies, how many scenes show you step by step how to make a Molotov cocktail? So true. It's not like the, the, the, the knowledge is not out there. Right?
18:10Matt Coles It's when it comes to an A-bomb, that's when you have a problem. That's, you can't find that.
18:13Izar Tarandach That, that too. That too. But, uh, I actually draw the line at the, uh, uh, fusion reactors in the backyard. But, um, or so the knowledge is out there, right? So going back to, uh, going back to models again, it's not like, how do I make this bad thing so that I can use it in my school project? It's about having the resources to actually do the bad thing.
18:38Matt Coles By the way, My Science Project, I think it is, it's bad '80s from the movie. Well, a really good '80s sci-fi movie where they built a nuclear bomb.
18:51Izar Tarandach I thought that you were talking about the one where—
18:53Chris Romeo My Science Project.
18:54Matt Coles I think it's My Science Project.
18:55Chris Romeo I think that's the one. You're not talking about Weird Science.
18:57Matt Coles No, not Weird Science.
18:57Chris Romeo It's the best science related—
18:59Izar Tarandach Weird Science is the best science fair movie ever.
19:02Chris Romeo Yeah.
19:03Izar Tarandach Okay. And that's the best school project that you can actually have.
19:06Chris Romeo But, uh, so people will have to look up the, uh, people have to Google the, uh, the plot.
19:14Izar Tarandach But the, the, my point is that, uh, and then Real Genius, of course, but Real Genius, Real Genius. Oh, Real Genius is the one of those. Yeah. So we, we, we, we have all this, this knowledge available. LLMs may just be making things, uh, uh, easier to find. But for our intents and purposes, the searching of bugs and whatnot, it's not like the older models can't do that. It's just that you have to point them in very, very, very pointed ways to go and find stuff, right? The new ones, they are more quote unquote dangerous because apparently they, you are able to say, hack this thing, make no mistakes. They do have a lot more, for the lack of a better word, agency to go and search for issues.
20:06Chris Romeo But what's the point of these guardrails in the US-centric models if an attacker can just go to China and use KIMI or one of these other things that have no guardrails? Like, why are we limiting? Because you're limiting defenders. In the same way that you're limiting attackers?
20:24Izar Tarandach I'm gonna go with security theater.
20:30Matt Coles The Manhattan Project was the name of the movie, by the way.
20:33Chris Romeo Oh yeah, yeah. 1986.
20:36Izar Tarandach Which is very different from Oppenheimer.
20:38Matt Coles But, um, yes, very different.
20:40Izar Tarandach Yeah. But, uh, yeah, so I'm gonna go with security theater. I mean, first they make you feel bad because, oh, these super weapons are out there and they're going to end you. Humanity as we know it, which may not be a bad thing. And then they come with, oh, we saved everybody because now we have these guardrails in place. And no, if you're not American, you can't use this because America, right? So yeah, I'm going to go with security tier. In the, in the end, it's what, what you're saying. It's just slowing people down. And we have seen a bunch of very smart people working with the resources that we already have. doing very good work.
21:19Chris Romeo So what happens if Anthropic and OpenAI just open the floodgates here? Do we have like a, a 30-day sprint of a nightmare or a nightmare sprint where there's so many bug reports coming in so fast of critical vulns that open source maintainers are overwhelmed, commercial software companies can't keep up. And there's a Wild West period where vulnerabilities are being exploited and without any fixes. Is that what would happen if they open this up?
21:53Matt Coles I mean, that's what, that's what the, that ACRITIS Foundation is for, right? The Linux Foundation ACRITIS project is to try and get ahead of that because there are so many vulnerability reports coming out of these models. If you open the floodgates, if you open the floodgates and make it economical for folks, bad actors or others to use these models, right? Because you still have to pay for those. You have to still make use of them. You have to potentially pay a subscription or have hardware. Most, I guess, it sort of goes back to the discussion that we've had probably years ago around. So you find a vulnerability, what's the likelihood of exploitation? Yeah, right. You have, you have a need for, there's a need for the attacker, the would-be attacker. And I'm not even, let's not even say the attacker, the would-be attacker that when you're, when you're doing that risk analysis, somebody has to have means, motive, and opportunity and intent, right? They have to have the ability, the reason and the desire to do bad things. So is somebody— if you get access to Fable or Mythos or whatever as a general purpose model, or even, you know, the KIMI thing or whatever, are you immediately going to go out and find exploitable vulnerabilities to exploit?
23:27Chris Romeo I mean, some percentage of people would.
23:29Matt Coles Yeah, some percentage of people would, and every government probably would, just because that's, that's going to be the, that's the, the new self-defense mechanism.
23:36Izar Tarandach Wait, wait, wait, wait, wait a second. We, we are looking at the thing, but we are not looking at around the thing. Okay. So the thing is models find vulnerabilities. Fine. Established.
23:48Matt Coles They can be used to find vulnerabilities.
23:50Izar Tarandach Yes. Yeah. Yeah. Okay. They do.
23:53Chris Romeo Cool.
23:53Izar Tarandach The around the thing, and going to what Matt said about the motive and the means. First of all, not everybody is going to have the money to spend on these models, finding those things, because it is an extensive search. It's not, hey, give me a vulnerability. So first of all, you have that. Second, assuming that it is, that they are models that are run by some framework or structure that actually cares about it, there is a very large extent of observability on top of what are people doing with my model. So chances are that you're going to trigger something, not a guardrail. You're not going to be told, oh, I can't do that. But some alarm is going to ring somewhere because words around the discovery of a vulnerability are going to be emitted. and a simple grep is going to find them. We are not talking anything very sophisticated here, but it's going to ring some bells.
24:57Matt Coles You mean it's noisy?
24:59Izar Tarandach It's noisy. It's noisy as F. But then there is also the actual information warfare theory of the thing here. The fact that you found the vulnerability doesn't mean that you go out and you use it immediately.
25:15Chris Romeo Yeah.
25:15Izar Tarandach These things have a very valuable shelf life.
25:18Chris Romeo Mm-hmm. So you keep it for when you actually need it. I'd argue less valuable in the age of these new models.
25:25Izar Tarandach Because everybody else could find it. Right. And also use it and then you're screwed.
25:30Matt Coles But, well, or, or report it. I mean, or report it, right? Because they could get money from the reporting.
25:35Izar Tarandach No, but it, it goes back. The fact that it's reported doesn't mean that the, the thing is invalid.
25:41Chris Romeo I got a strange—
25:42Izar Tarandach So wait, walk with me here. So I'm somebody who found a vulnerability, okay? Very, very high-profile one, very, very able to enter into very interesting places. Now, if I'm going to exploit it right now, I have to find a way to mask myself so that there's no coming back to me. I have to find a way to actually exploit it at the places that are going to be most meaningful so that I get the most bang for the buck, I have to decide on a number of things that would make the using that valuable to me.
26:18Matt Coles Now, that's if you want to derive value from it, but you could just drop it into a script and let us know.
26:23Chris Romeo This is worm central, what he just described.
26:26Izar Tarandach Wait, wait. Now, if I wait 2 weeks and other people discovered it and there are a bunch of script kiddies and they did the worm thing and they started exploiting it everywhere, All of a sudden, I don't have to put so much into protecting myself because I'm going to be lost in the noise. We do know that these things do not get fixed initially. It can take, what is it, 6 months to a year to fix a critical. So it's going to be much easier for me to actually exploit that thing.
26:52Matt Coles I mean, if it ends up on the campus or something, it's going to be, you know, 7 days.
26:58Izar Tarandach That actually being in the first.
27:00Chris Romeo Now there's a lot of value to be in the noise.
27:03Izar Tarandach Exactly.
27:04Chris Romeo That's almost, you think of that as a strategy. I'm gonna, I'm gonna release this vulnerability in a way that other people can find it and start exploiting it so that I can hide in the noise.
27:14Izar Tarandach Or you find two, you release one of them, and while everybody's looking at the left one, you use the right one.
27:21Chris Romeo Dude, this is going in my book. Hold on, I gotta stop and take some notes. Hold on. That's brilliant.
27:26Izar Tarandach The, the, the thing is there is a whole, uh, uh, uh, almost game theory. Point to the thing here, that it's not just figuring out the vulnerabilities, when it's going to be used, when it's going to be exploited, what's the value that's going to be obtained from doing it. Now, my point is that nothing tells us to this point that sitting in North Korea, there isn't a USB drive with 100 of these in there, and they're just waiting for the right time to use it.
27:53Chris Romeo Yeah, but come on. There's, there's I'm not going to go there. I was going to say something that could have potentially gotten me in trouble. Um, after I've been listening to you go down this train of thought, I had a conclusion hit me and I'm curious if you guys agree or disagree. The new models that we're seeing now spell the end of bug bounty.
28:18Izar Tarandach I think that one very interesting thing is that bug bounty is the first frontier where AI is going to fight AI. For actual vulnerabilities.
28:27Chris Romeo Does it eliminate a company paying an individual some number of thousands of dollars for finding a critical level vulnerability? Because can't you just have the AI, your Mythos, look for it in your codebase and you don't have to do it publicly anymore?
28:44Matt Coles So, I think to some extent—
28:45Izar Tarandach Your book.
28:46Matt Coles I was just saying, I think to some extent, yes, you don't have— a company may not need to pay have a bug bounty to pay, you know, to pay for people to find issues. Um, alternatively, well, okay, it may depend on the size of the company, right? So arguably there's a, there's a threshold, a floor for the size of a company that can pay for bug bounty in the first place.
29:13Chris Romeo Exactly.
29:14Matt Coles Right. And then those companies, there's may, it's probably a slim band where it's cheaper for them to pay bug bounties than to run the models to do the bounty themselves. But at some point it's gonna be more cost-effective.
29:29Chris Romeo For the individual, I, I just thought of this, like the individual is gonna have to pay to use a model to do what people would've done by hand in the olden days, the golden age of bug bounty.
29:41Matt Coles And they may not get the same payout that makes it economically feasible.
29:44Chris Romeo You'd think the payouts would, would shrink.
29:48Izar Tarandach Alternatively, the payouts would probably shrink.
29:50Chris Romeo Yeah.
29:51Matt Coles Alternatively, the bug bounty parameters would change to reward the human in the loop part of the discovery, right? So, maybe the human is utilizing the AI models for something, but now the report has to be much more precise and maybe it needs some human impact or needs a business impact as opposed to just a vulnerability line item.
30:15Chris Romeo There's almost like a business case to be had of, should I do a bug bounty as a researcher? Because how much is it going to cost me in model, in usage charges versus what is the if I found a critical vulnerability what's the reward? I have a feeling those are going to very quickly—
30:34Izar Tarandach it's kind of like Mike Wayne, wait, we are looking at the thing from the very simplistic point of view that again the researcher sits in front of the thing that's being mounted and goes find me a bug make no mistakes that's not how it works right that's where you get the high spend.
30:54Matt Coles But if they use—
30:58Izar Tarandach that's a t-shirt. If they use it instead as a tool to help them in their workflow, if they integrate it into their workflow, then the spend in tokens is going to be much smaller because they are applying the experience that they already have in reconnaissance, in figuring out the threat model, in figuring out the attack surface. Okay.
31:20Matt Coles And then they can expand that or, uh, what is it, normalize the cost over the, distribute the cost over the bounties that they—
31:28Izar Tarandach So if you attack the thing with a, uh, uh, what's the name of the, uh, the electric saw, it's going to cost X. But if you go with a scalpel, you're probably going to get much better results and it's going to cost much less.
31:45Matt Coles Now there's the other side of the equation. More efficient, more economic.
31:49Izar Tarandach Now there's the other side of the equation.
31:51Chris Romeo A recommendation you should always make to your doctor. Indeed. Go with the scalpel, not the chainsaw.
31:55Izar Tarandach Unless you're doing an amputation. But the, the, the other side of the thing is the, the, the inside of the, of the bonding program. The moment that they start getting scalpeled by like 1,000 people where before they had one person. Now they have to deal with that 1,000 reports and figure out if they are valid or not. And we have been seeing a lot of work done in AI with doing the validation of the bug. I won't say the exploit because if it exploits, then it's valid, but the validation of the report. And that goes into reachability, that goes into observability, that goes into the whole understanding the chain of events that leads you. From that open port or that endpoint to somewhere in the code where bad things actually happen. So the bug bounty now is going to have to put much, much more automation in place that otherwise would sit with blue teams.
33:03Chris Romeo Yeah.
33:04Izar Tarandach To check out.
33:04Chris Romeo So there's almost like an AI validation step. Needs to be added to the first seed.
33:08Izar Tarandach And that's where I say that I think that, that's where I say that bug bounties are probably going to be the first arena where we're going to have AI against AI with bugs in the middle.
33:18Matt Coles Right.
33:19Izar Tarandach So that's probably going to generate a very good industry of verification that at the end of the day, everybody's going to be able to get something from.
33:27Matt Coles Is it going, do you think it will change, like traditional bug bounties, I guess, I think traditional bug bounties were really more about the vulnerability itself and less about the impact of that vulnerability. Do you think that in the future it will be more about the risk that's mitigated or the risk that's identified or the breadth of the impact and more so than just, is this a vulnerability in code?
33:58Izar Tarandach I think that for a long time, bug bounty teams, like not the ones inside companies, but the ones that you hire, they have had this very limited, uh, uh, uh, view of the thing that they are protecting. So they know if something works or doesn't, right? If something gets in or doesn't, if it's bad or if it isn't. But they don't have the whole vision of saying, okay, this is how bad it is for the business. This is how bad it is for the system. This is how bad it is for the long term. So that kind of impact is still something that lives inside the company that's hiring the bug bounty service.
34:39Matt Coles So to make the bug bounty valuable, to make the bug bounty valuable, in other words, to justify the payout, it sounds like that would need to be introduced into the conversation. In other words, the bug bounty clearinghouse may no longer be— maybe the bug bounty clearinghouse may no longer be necessary, like is Bugcrowd. Or HackerOne required now to manage a bug bounty if you have AI that's doing the intake and doing the validation and you can tie it to your risk management platform.
35:08Izar Tarandach Right. So the, the, the bug bounty clearinghouse, the economics of their, of that were that not every company can build a team that's able to replicate what they do.
35:19Chris Romeo Right.
35:19Izar Tarandach The problem is that now all of a sudden we can approximate. That's pretty wild.
35:25Chris Romeo So I asked Claude about the increase in bug bounty submissions, and submission volume is up dramatically, in some cases 2 to 10x. The signal-to-noise ratio is down enough that several programs have paused or shut down entirely rather than deal with the flood.
35:43Izar Tarandach Has it given you a reference?
35:47Chris Romeo Um, there's about 7 or there's a reference from each bullet point. Um, okay. Bugcrowd. Bugcrowd said submissions more than quadrupled during a 3-week period in March 2026. So Bugcrowd's, you know, between them and HackerOne, they pretty much own the, the bug bounty platform space.
36:09Matt Coles So bringing it back, can we bring it back to the Linus article?
36:12Chris Romeo Just to do it. Oh yeah.
36:13Izar Tarandach We started with that. Yeah.
36:14Chris Romeo Please find a way to get back there. Love to hear how you're going to do it.
36:18Matt Coles Just that AI is a tool that is now useful, and that it has— it is put— it's put on par with the human effort.
36:28Izar Tarandach And just to cross that with the bug bounty verification thing, do not forget that he's not talking on a broad sense. He's talking about the Linux kernel here, which is one of the pieces of hard— of software out there that actually does write the thing that we have been saying all these years, that you have to review code before you check it in. So it's not like, oh, it's AI, let's just check it in and see what happens. No, there's a very, there's a very extensive, very rigorous process of vetting the code that comes in here. So he's talking about, he's talking from a position where he doesn't suffer what 99.99999% of other places does. Which is, hey, it's AI, let's put the code in and see what happens.
37:12Matt Coles You know, and he's, he's in the same boat as folks like the BSDs, right? OpenBSD and FreeBSD. Right. They have a very strong release engineering process, a rigid, right? A structured way of how, how changes are made. And, and it shows in terms of the number of exploitable vulnerabilities and things. So I guess he's just saying, I guess what he's just saying is AI is a tool like your IDE used to be. And to some extent with the little dig at the human intelligence or natural intelligence that remember when developers used to use Stack Overflow to figure out stuff and how—
37:53Izar Tarandach Now they use Modebook.
37:54Matt Coles And how fallible that was. And so it's okay if you use AI because we're doing it and if you don't want, if you wanna be a purist, go have at it.
38:08Chris Romeo That's a good, yep. We've pretty much solved life, the universe and everything.
38:16Matt Coles And now it's just a matter of waiting for the book to come out in all its infinite dimensions. And, or, or to see the bill that Chris gets at the end.
38:26Chris Romeo Yeah. Let me check and see. Oh, this isn't good. Oh wow. This actually, this isn't bad.
38:34Matt Coles That's the scary thing. It's not bad compared to like, just remember you're, you're just, you're only stealing from everybody who else, who wrote a book out there that it got trained on.
38:43Chris Romeo Listen, that's not my problem. I didn't do the training. Okay. It wasn't me. I'm just a New York Times bestselling author, self-proclaimed because self-proclaimed you will, you will.
38:57Matt Coles And Claude will put it there in the byline for you.
38:59Chris Romeo Oh, I hope.
39:02Matt Coles You have to have it write a good forward and a good review page.
39:06Chris Romeo Oh no, I'm going to have it write, have you write the forward for me.
39:12Matt Coles I'm just going to use Gemini or something.
39:14Chris Romeo Have you created a skill that says speak like Matt Coles does?
39:21Matt Coles I should.
39:22Chris Romeo I've done it on Claude. All you have to do, or if it—
39:26Matt Coles Wait, you honor me, sir, that you have created a skill for me.
39:31Chris Romeo I have, cuz I want to just get Matt's take sometimes when I'm at it and I'm dealing with something. I'm like, what would Matt do?
39:38Izar Tarandach And I hit the button and I don't know, I'm just counting down the moments for it to say it's not Matt, it's César.
39:44Chris Romeo It's we, we are the Borg. Well, I think we covered quite a bit of ground there.
39:57Izar Tarandach Might have been a big circle, but yeah, it often is. I've noticed good things, good things are circles, spirals. They go up, they go up.
40:06Chris Romeo But yeah, it's kind of a spiral too.
40:08Izar Tarandach Yeah. All right.
40:09Chris Romeo Well, folks, thanks for tuning into the original, not to be confused with cheap competitors. Security Tales.
40:16Matt Coles You mean the forks?
40:18Chris Romeo The forks.
40:19Izar Tarandach The forks.
40:19Chris Romeo Our podcast has been forked. The forks.
40:23Izar Tarandach I feel forked.
40:24Chris Romeo Linus's statement here, we could send it to our other, other podcast.
40:29Matt Coles Well, they didn't like it and they forked it.
40:32Chris Romeo Yeah.
40:33Izar Tarandach Hey, people are more than free to not like it.
40:38Matt Coles Just not fork it.
40:39Izar Tarandach But on the other hand, do you have two listeners, one in Australia, one in Brazil?
40:45Chris Romeo That's right. We are big in the South. We're international.
40:49Izar Tarandach We are big in the South.
40:52Chris Romeo I'm going to put that now on the description. Internationally recognized.
40:57Izar Tarandach Exactly.
40:58Chris Romeo We have listeners in 2 countries.
40:59Izar Tarandach We are big in the South. Yeah.
41:01Chris Romeo We're big somewhere. I don't know. Still waiting. The jury's still out as to where that is. I don't know. I gotta check the stats again.
41:10Izar Tarandach See how we're doing in Australia. We're going to have to start opening with Bom dia, mate. So that we can cover both.
41:16Chris Romeo Yeah.
41:16Matt Coles Yeah.
41:16Chris Romeo If you could just give me some cue cards or something you could hold up during the opening. All right, folks, we've had enough fun. Go out there, uh, just gonna say fight, but yeah, spend some tokens, build some cool stuff, get your monthly recurring revenue to over $1,000, something like that. I don't know. Find the future. Look to AI.
41:42Izar Tarandach Matt, post on TikTok, do something.
41:44Chris Romeo No, don't post on TikTok. It's actually still owned by the Chinese government last time I checked.
41:49Matt Coles Oh, true.
41:49Chris Romeo Yeah. So don't do that. This is a security podcast, but, uh, we just can't find a way to say goodbye. So I'll do it. Goodbye. Bye.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
