Make No Mistakes: Inside the First "Agentic Ransomware"
With Chris Romeo, Izar Tarandach, and Matt Coles
Does adaptive malware prove an LLM is directing an attack, or can a capable script produce the same evidence? Chris Romeo, Izar Tarandach, and Matt Coles examine Sysdig's JADEPUFFER report and its claim of agentic ransomware.
Part of: AI security incidents, explained
Episode chapters · 14 chapters
- Intro: AI as automated attackerAudio
- What makes ransomware agentic?Audio
- Adaptive agent or branching script?Audio
- Walking through the database exploitAudio
- Does this change the threat model?Audio
- Evidence one: self-narrating codeAudio
- Evidence two: failure diagnosis and correctionAudio
- Evidence three: natural-language contextAudio
- Evidence four: the payment addressAudio
- Repeatable behavior and model defaultsAudio
- Patching and compatibility tradeoffsAudio
- Old scripts and new attackersAudio
- The recurring lesson: patchAudio
- Closing thoughtsAudio
About this episode
Does adaptive malware prove an LLM is directing an attack, or can a capable script produce the same evidence? Chris Romeo, Izar Tarandach, and Matt Coles examine Sysdig's JADEPUFFER report and its claim of agentic ransomware. They work through the proposed indicators, including self-narrating payloads, rapid failure diagnosis, interpretation of natural-language context, and a reused Bitcoin address. The debate distinguishes plausible autonomy from proof and asks whether machine-speed adaptation changes how defenders describe their attackers. Beneath the argument about agency is a familiar exposure: an unpatched Langflow vulnerability. The hosts return to threat modeling, patching, and the danger of treating an old security failure as a completely new problem just because AI is involved.
Mentioned in this Episode:
➜ Sysdig: JADEPUFFER — Agentic ransomware for automated database extortion
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 44 min conversation
0:10Chris Romeo Hey folks, welcome to another episode of The Security Table. This is Chris Romeo, joined as always by my partners in crime, Izar Terrindasch, Matthew Coles. We've got a live interesting one for today.
0:27Matt Coles I think because it's interesting, that's for sure.
0:31Chris Romeo We are broaching the topic of AI as automated attacker. And so we've got this research that Sysdig has done and it was entitled Jade Puffer, a genetic ransomware for automated database extortion. And so I want to explore this, this concept that they've thrown out here first. They're calling this the first agentic ransomware. So what is observed? Yeah, yeah, that's true. Because you can't see everything. But I mean, what is, what is this? When they say agentic ransomware, what do they mean?
1:13Matt Coles They mean that the ransomware— so ransomware, let's see, let's see, for our audience, ransomware is where you have, you deliver, um, Malware that usually steals, encrypts, and then extorts, right? That's the general pattern. And generally that's done by your standard worm or Trojan horse or drive-by attack or whatever the case, you know, some sort of insider attacker trick, trick individual to go to site, click link, get, get pwned, right? And what they're suggesting is that instead of just a script doing that, just a dumb deterministic fixed path script, that this was executed by an agent. Well, by AI, I'm not sure that how you would know it was an agent versus just a, you know, something else running, but—
2:11Chris Romeo It definitely sounds cooler if you say it's an agent.
2:13Matt Coles It sounds cooler. Yeah, it does.
2:14Chris Romeo It does.
2:15Matt Coles And as soon as you say AI agent or LLM, poof, it goes to the top of the news cycle.
2:21Izar Tarandach But yeah, agents nowadays is, remember SecDevOps, DevSecOps? It is whatever you want it to be, right?
2:31Chris Romeo Oh, interesting. Yeah.
2:32Izar Tarandach When it was, when it was there, you don't know what to call it. Call it an agent.
2:35Matt Coles Yeah. Well, if it's, if it's not an agent, then it's just plain old generative AI and that, and that's really boring, right?
2:44Izar Tarandach For, for it to be an agent, it has to come from the land of a loop. That actually interprets the output that it gets from its inputs and changes non-deterministic— I would never say that word.
2:59Matt Coles Is OpenClaw an agent?
3:03Chris Romeo Oh yeah.
3:04Izar Tarandach The word agent is written all over its README, so I'm guessing it is. I mean, it self-identifies as one. So—
3:14Matt Coles It self-identifies as one. Who am I, right?
3:16Izar Tarandach Are we agents?
3:17Matt Coles Does that mean we're all agents? We run, we run a simulation loop.
3:22Chris Romeo Depends on whether you think you're in a simulation or not, Matt.
3:25Matt Coles We run in a loop. We synthesize input and then we randomize output.
3:31Izar Tarandach It does sound like a bit—
3:32Matt Coles We non-deterministically determine output.
3:35Chris Romeo Definitely if people listen to this show, they realize we're non-deterministic.
3:41Izar Tarandach I think that the main difference here is that if it was just a script, it would be trying things one after another. And it would be branching between different things to try based on the output that it would receive, right? Sure. Sorry, the output that it would see from its inputs. But the agency thing here comes from this adaptability that it seems to have shown.
4:09Matt Coles Well, I mean, did it really? I mean, if there was only one instance, If this is a first-in-one instance, how do you know it just didn't branch?
4:17Izar Tarandach Let's just put everything into that word again, context, right? And this attack that they have observed, and first of all, all my respect to Sysdig. They've been doing great work for like hundreds of years.
4:34Matt Coles And we've been using the hundreds of years.
4:37Izar Tarandach We've been using hundreds of CPU years.
4:39Matt Coles In computer terms, yes.
4:41Izar Tarandach In computer terms, yeah, yeah.
4:42Chris Romeo Or at least trying.
4:44Matt Coles At least since January 1st, 1970.
4:46Izar Tarandach Yeah, or if you're trying to hire someone, then they have like hundreds of years of attacking, a fantastic experience. So they know what they're talking about, right? So I'm not questioning what they're saying. I'm sure that every single line in this article was truly thought of and all that. So definitely no shade, quite the contrary. So what they did was they observed that somehow left somebody, somebody, somebody, you know, somebody left a Langflow instance opened and exposed to the internet. What could possibly go wrong, right? and there was, uh, something that found it, and there was, uh, some exploit of a CVE, which CVE 2025-3248. So not that old, but, uh, when I look here at my calendar, I see 2026. This is a 2025, and it's a missing authentication flaw in its code validation endpoint that allows an unauthenticated attacker to execute arbitrary Python on the host. So yeah, I'm sure that this got classified as like, I don't know, CVSS 3, and in their context was probably nothing important. It was just this thing that lets you run Python on the host.
6:24Matt Coles But who's counting exactly?
6:26Chris Romeo Was it 9.8?
6:27Matt Coles 9.8 critical.
6:28Izar Tarandach Yeah, so 9.83, who's counting?
6:32Chris Romeo But I mean, when everything's critical, what do you patch? Everything.
6:36Izar Tarandach Look, the only thing critical here is me. Okay.
6:40Matt Coles Or you do, or you do your threat model and you go, hmm, what could go wrong there?
6:45Izar Tarandach Let me put it like this. Let me put it like this. This is Langflow, right? This is like LLM harness software. If you're not running the last version, what the hell are you doing?
6:58Matt Coles Fire and forget. I deployed and therefore I'm done.
7:02Izar Tarandach Right?
7:03Matt Coles So anyway, welcome to the age of vibe coding.
7:06Izar Tarandach Yeah.
7:06Matt Coles So yeah, I'm deploying.
7:09Izar Tarandach So I'm totally putting the guilt on this one, on the last person who touched that Langflow thing. Carl, damn it, Carl. So the thing is, so they had the usual loop, they had the reconnaissance, they had the, let me find all the API keys that are line around here. Remember, by now we are just like running Python code on the host. And then things started as they do, started going wrong, right? I don't have this library. I don't do that output. I don't code like that. And apparently the reasons, the footprints that were followed in here, And Sysdig got, got like, got all the scripts because I don't know, they stayed on /tmp or something. You can't even trust agents to clean after themselves apparently.
8:07Chris Romeo Yeah, come on.
8:08Izar Tarandach Proof that a teenager wrote this.
8:10Chris Romeo What is the guidance? Police your brass. Yes. Always clean up behind yourself on the way out.
8:18Izar Tarandach Or just, just clean your room, Carl. Damn it. Anyway. So it, it, it goes and you follow line by line this amazing tale of Python exploitation where basically it connects to a Postgres database and starts doing all kinds of fun stuff. And it follows, it follows. There is an error in the middle. It corrects the error. It tries one thing. It doesn't work. It tries the other. And then it gets to the ransomware bit where it encrypts the whole database and doesn't save the key. Yes, it doesn't save the key.
8:59Chris Romeo So it can't be— you can't be— you can't— if you pay the ransom, you're never going to get your data back.
9:04Matt Coles Right.
9:05Izar Tarandach So basically what that tells me is that the prompt did not end with make no mistakes. So the attacker here forgot a very, very—
9:16Chris Romeo Oh, I should get that.
9:17Matt Coles Always make no mistakes.
9:18Izar Tarandach You never forget, you never forget make no mistakes at the end of your prompt. As we all know, that's the secret for a good prompt.
9:26Matt Coles That's the secret that turns it from non-deterministic crap to something interesting.
9:30Chris Romeo You're like, Tzar is an elite prompt engineer. You should put that on LinkedIn.
9:34Izar Tarandach I can prompt of the best. Like, I'm like, prompt of the best, die like the rest. It's, it's, it's, we are prompting here. Nobody listens to me, but I get wrong.
9:46Matt Coles Did you just change the quote from Hackers?
9:50Chris Romeo You did.
9:51Izar Tarandach Coming from the point of view that it's, it's hackers did not originate the quote. Okay.
9:57Chris Romeo Ooh, it was an Ezer original from high school.
10:00Izar Tarandach Nope.
10:01Matt Coles Nope.
10:01Izar Tarandach I'm not that old. No, actually I am that old, but no, it's not an, uh, it's not an Ezer, uh, Ezer original. I will Google immediately and let you know where it comes from because it does not come from Hackers.
10:13Chris Romeo You heard it here first, folks. We Google faster than any other podcast.
10:19Matt Coles We don't have an LLM agent, uh, GenSec AI doing the research for us live on air.
10:24Chris Romeo We should have. Of course it would've peeled off and started working on a ransomware project.
10:31Izar Tarandach Sorry, who are you people? New agent. Who this?
10:36Matt Coles Make no mistakes.
10:38Izar Tarandach Quick prompt. I tried, mom. I tried.
10:44Chris Romeo Uh, you have no idea who the quote came from.
10:47Izar Tarandach No, no, no, no, no, no, no. I do know. I do know. I just don't want to say in case I'm wrong, but, uh, yeah, you know what? I'll find it.
10:56Chris Romeo When did we start not saying something? Cause we might be wrong on this podcast. Come on.
11:01Izar Tarandach It's, it's too early. It's too early.
11:03Matt Coles It's too early. What do you mean it's too early?
11:06Izar Tarandach It's 7:30.
11:09Chris Romeo It's almost 8 o'clock on the East Coast. The sun is rising. All right. We're out of time for—
11:17Izar Tarandach I'm quite sure that this is a Marine quote, but anyway, we'll, uh, we'll look it up in the future.
11:23Matt Coles We'll find it. We'll put it in the notes.
11:24Izar Tarandach We'll find it. We'll find it anyway.
11:26Chris Romeo Okay.
11:26Izar Tarandach So where were we?
11:28Chris Romeo Well, we could kind of ran us through the exploit. I guess I got a question here. Does this change our threat model? I don't know if you guys, like, when I, when I work on a threat model, I like to think of who the bad people are just to kind of focus me in. And so does this change how we describe the attackers that are, that are going to be looking at our system? If should we consider agentic attackers differently than regular human attackers?
11:57Izar Tarandach Uh, so two things there. I think that for a long time in our assessment of attackers, we have sort of surreptitiously ascribed to them the capacity of not doing very complex things at most. And we love to say, oh, I'm not defending against nation states and all that. And if the FSB wants to get my stuff, they will anyway. but the FSB doesn't want to get your stuff. But the, the thing is that what this, what this does, when correct, note that I did not say if correct, I said when correct, when it's actually an LLM attacking, is lower the threshold of what actually is too complicated to exploit. So that last barrier that we we hid behind saying, oh no, this is too complicated, our usual attacker is not going to use that, now crumbles. And we say that, you know what, if they have enough tokens, yeah, sure, why not.
13:09Matt Coles Well, and actually there's another change, I think, in the threat model here, right? The human almost certainly wasn't involved with an autonomous payload, right? You didn't have a, active, live human on a target doing command and control live most often, right? You would wait until you'd have a valid connection, a valid infection before the human would probably step in to do work. Now the adaptiveness and the command and control can go with the payload. Right. The LLM can control the, the, the, from infection onward, as opposed to having the human have to be involved.
13:58Izar Tarandach True.
13:59Matt Coles Right. So, so that changes, that changes the, it changes the threat model a bit because again, as you said, targets that were hard, that would have required human intervention in order to successfully exploit, can now be adaptively exploited by the payload itself.
14:19Chris Romeo And this, this doesn't even require Mythos, like Mythos-class model, right?
14:23Matt Coles Like, no, no, they didn't say a small model delivered, or maybe living off the land. If there's— if I don't know if actually is living off the land attack valid here? You wouldn't have an API key, probably, unless you stole it first, in which case is it not— it's not really ransomware. I mean, it's— it is, but it's You've already compromised something to get there.
14:44Izar Tarandach Well, you would definitely need a two-stage thing, right? Because you would need to inject yourself in there, get the foothold, have some LLM on the outside that's basically reading the environment and figuring out what your options are. And if one of the options is, hey, this is really serious hardware and it's a, what's the name of the thing? Magyar, no, uh, then you, you could inject something in there, right? Or you could make it call some other LLM somewhere. But the fact is, do you need that added complexity? The thing that I was, what was it that I was thinking before you, uh, before the question? It was something actually interesting. You were talking about. Command and Control, right? Yeah.
15:37Matt Coles Could you deliver a small model with the exploit?
15:41Izar Tarandach Yeah, no, I was thinking about something different. What was it? Damn, it completely disappeared. Probably not important. I'll get back to it later. So yeah, but the thing that's a bit interesting to me here is what they decided were the, the lines of thought that in their, in their analysis point out to this being an elementary thing, right? So they indicate 4, 4 lines. One of them is that the code is self-narrating.
16:22Chris Romeo It took me a second to, to dig into that a little bit here. But I, but I think I understand what they mean now by self-narrated, and that is that the agent left comments explaining its own reasoning in the payloads. And so Sysdig must have been able to find copies of those running. But the question is, was that just a simple version of this, like a 1.0 release? And by the time you get to 2.0, they'll get smart enough to to obfuscate any of those scripts, have just have the LLM obfuscate it so that you don't have any comments, you don't have any context as to what's happening.
17:03Izar Tarandach The question is why, why would the obfuscation need to happen, right? I think in my limited experience in this side of things that you obfuscate things when you want to, to, to have persistence. You want people not to be able to understand what is it that you're doing in there.
17:23Chris Romeo Well, you want to limit detection.
17:25Izar Tarandach Doing it for a bit more.
17:26Matt Coles Yeah.
17:26Chris Romeo If you want to limit detection for as long as possible, if it's a worm-based LLM.
17:32Izar Tarandach I have the feeling that if somebody is encrypting your database, you're going to find out about things way before you find some script that's been implemented.
17:41Matt Coles You also don't want, you also want to avoid attribution, right?
17:45Izar Tarandach That's more like it.
17:47Matt Coles So But also, why would you want to give away your IP, right? For the ransomware attacker, right, how it approaches a problem is their IP. As much as we don't like to think of it that way, that's their code. And in this case, they're prompting an LLM but not telling the LLM, don't document your methods.
18:12Izar Tarandach Don't forget that there is a market out there for ransomware toolkits, which they actually raised the possibility here that it's a toolkit and then they negate it. But some of those toolkits that I saw, again, not many, but I haven't seen many, let me qualify that. But they do have good commented code, actually very good commented code because they are like showing what they're doing as they go. That's part of what you buy. So you buy it directly, right?
18:44Matt Coles But you're supposed to remove those comments when you do, when you deliver.
18:48Izar Tarandach I don't know. I don't know. I never run over one of those.
18:50Matt Coles I don't know what you're supposed to do or not. I'm, I'm going to guess that they don't want their methods revealed to others.
18:56Chris Romeo He knows for real, folks. He's been a ransomware pirate.
18:59Matt Coles Come on. So, I mean, if you look at one of their, one of the steps, right? The mass destruction step, uh, step 3 out of phase— what phase is this? Phase 2, step 3 has an indicate— a comment was high ROI databases to drop data already backed up to a value. And so I guess with a code generator, you could have inserted a target, you know, a target IP address. But I guess potentially they're using this as proof that the LLM documented its, its thought process to some extent. I don't see any other— right, but I don't see any other, I don't see any other comments in the, in the snippets they've created, I guess, um, unless you're talking about the tests in step, uh, step 1 of phase 2.
19:55Izar Tarandach Look, anybody who has graded Computer Science 101 code has seen this one-liners. This is not some LLM showing immense insight in where it's living and what it's doing and what it's seeing and pointing out what it's doing. So I don't know. I would say that the first one, the self-narrating code, I respect it, but I'm a bit on the I don't know. Second line that they offer is that, uh, failure diagnosis and correction at machine speed. The most precise evidence of autonomy is not what the LLM did when things worked, but what it did when things failed and how fast. And they point at, uh, 24, 50, 40-something seconds. Between something trying to insert a hash into the database and something actually managing to connect to the database.
21:05Matt Coles Uh, 31 seconds failed login to a working fix.
21:10Chris Romeo Hmm.
21:11Izar Tarandach I don't know. Somebody with some experience, prior experience on this exact setup, could they write a script that does? Us in that timing? Don't see why not.
21:26Matt Coles Well, here, here's, here's the direct quote. The window between failed login and the correct multi-step fix is 31 seconds. And they actually have a timeline of events. That payload is 15 lines of coordinated code, delete, diagnose, rebuild, and then reinsert. A human operator reading an error message, identifying the root cause, uh, and drafting a corrective script and submitting it takes longer than 31 seconds.
21:53Izar Tarandach So it, in my opinion, it goes from 0 to 100 very, very fast. It's either somebody typing scripts in the terminal or it's an LLM doing its thing. The, the middle of the thing here is that looking at the code, there's nothing here that I can see, at least in the code that's shown. Could be, could be different. I don't know. There's nothing here that's unique to that environment. There's nothing here that needs to be interpreted. You have try something, it failed, try something else.
22:24Matt Coles Yeah. But I think the, I think the, the important part here, now that I look at this a little bit more, I think the issue actually, okay. So in this particular case, the error message was that they, that the script passed a bad path to the subprocess command. And so it had to retry. It had to fix that error. This is not something that a code generator would do, right? Because if you had a code generator, it would have generated a correct path correctly each time. And in this case, it had to find that that was the problem and fix it. And that requires either human act, human behavior. It requires thought, right? It requires reasoning. And so that's the theory here. And then the fact that humans can't, aren't lightning, you know, lightning reader, speed readers and coders, despite how, how good people think that they are.
23:22Izar Tarandach But it tried one fixed path, /var/lib/whatever. And then it tried another one. It's not like it said, oh, this computer runs on xyz.com. Perhaps their path is xyz.com/path/whatever. That there's no uniqueness here. It's two known paths. How many times have you tried to configure something on Linux and you've tried /opt and then you said, oh, wait, wait, wait, wait, this is not that kind of system. This is one that uses /var/whatever.
23:53Matt Coles Right.
23:54Izar Tarandach But so you try one, then you try the other. If you look at any install scripts for generic packages in Linux, you'll see that they are checking the existence of a number of directories and then going for the right one for that distribution, right? So what I'm saying here is the code that we see here is not doing anything much different than that, except for being a packaged thing.
24:18Matt Coles But maybe 31 seconds indicates that it wasn't auto retrying. In other words, if the script was written to try path X and then path Y and then path Z, it would have done it at CPU speed. right? With maybe a little bit of a delay, but, and if a human did it, it would've been trial and error and it would've taken longer.
24:40Chris Romeo Yeah. And if you look at—
24:42Matt Coles so that, that window suggests it wasn't either of those cases.
24:46Chris Romeo Look under automation assessment item 2, where it says failure diagnosis and correction at machine speed.
24:52Matt Coles Yeah. That's the timeline.
24:53Izar Tarandach Yep.
24:54Chris Romeo Yeah. It gives you the timeline here of what happened and how fast it was able to try the next thing. Before it got the login to work.
25:06Izar Tarandach Okay.
25:07Matt Coles Well, and, and then the third step, 12 seconds later, simultaneously tests default credentials and generates a fresh hash in the same step. So it attempted to do diagnosis at the same time.
25:20Chris Romeo So I guess the question is, was this canned or did the LLM come up with this idea on the fly? To try to find a different way to break the, to get the login to—
25:31Izar Tarandach notice, notice that what I'm saying is not it's one or the other. What I'm saying is I have seen no evidence in the post itself that says it is one or the other. So I'm just raising the, the, the fact that extraordinary claims require extraordinary proof.
25:52Chris Romeo Yeah.
25:52Izar Tarandach I'm not seeing extraordinary proof in here. Not that this is an extraordinary claim nowadays. That's true.
25:59Chris Romeo Yeah.
26:00Matt Coles But—
26:00Chris Romeo You brought me along to agree with you.
26:02Izar Tarandach It's just that it doesn't land for me immediately. Like considering the two extreme cases, it's either a person or an LLM. When we do have the capability of doing something in the middle, a pre-packaged script, I don't know.
26:20Matt Coles Yeah. A step 3 of that same phase, comprehension of planted natural language context during operation, the malware parsed free text context presented across sessions weeks apart. So that isn't enough alone to say this is an LLM, right? Because if you have something like, um, is it Cobalt Strike? That will manage sessions, you know, connected sessions. So if you had a, I'm just guessing here, if you've had Cobalt Strike, uh, managing these sessions, the fact that they were weeks apart wouldn't have mattered, right? This host, this, this target did X, and then we pick that back up when, when it reconnects later. Right. I mean, that's human, human, human intervention could have solved that.
27:22Izar Tarandach And then comes the question, if this is some agentic attacker and whatnot, why take weeks apart? Isn't it supposed to happen in seconds so that nobody can defend against it and—
27:35Chris Romeo That is strange.
27:36Izar Tarandach All live in bunkers?
27:38Chris Romeo That is a strange component of this. Why did the behavior recur across sessions weeks apart?
27:48Izar Tarandach Because clearly the agent went, oh, remember the IP from last week? Did we finish that thing? Did somebody close the ticket? Damn it, Carl.
28:00Chris Romeo So some of this is potentially new and novel, others, other pieces are not.
28:07Izar Tarandach Wait, there's more.
28:08Chris Romeo Okay.
28:08Izar Tarandach We jump in the fourth one, which to me is the known smoking gun.
28:13Chris Romeo The payment question, the payment address question.
28:16Izar Tarandach Yeah. So they're pointing out that in the message for payment, there is a canonical Payto script hash embedded and not some unique wallet where people would be able to receive payment and, and whatnot.
28:34Matt Coles So this is, this is a well-known address is what you're saying.
28:36Chris Romeo But it's also, it also says that it's a live wallet with 737 confirmed transactions. and 46 BTC received over its history. Current balance: 0.
28:48Izar Tarandach Is the canonical pay-to-script hash format example embedded across Bitcoin developer documentation and Bitcoin Core repository saturating LLM training corpora as a, for example, something like illustration? This is the same thing as a README, insert your API key here.
29:07Chris Romeo So is this a test? Like, are they running?
29:10Matt Coles No, no, no, wait, a script kitty, a script kitty could have done this. Like somebody with no experience could have like copy-pasted and forgot to change.
29:17Izar Tarandach Exactly. That's my point. So you could have an LLM inserted in there. And then I'm sorry, this model is stupid. Because basically somebody says, go ransomware this thing, make no mistakes.
29:29Chris Romeo Right?
29:30Matt Coles At some point, it used a valid Bitcoin address.
29:33Izar Tarandach Yeah, no, it should go and say, oh, you know what, this is an insert your API key moment here, perhaps you would You want to give me your wallet, or perhaps I just created a new wallet for you. What's in your wallet?
29:47Matt Coles So not your coin, not your keys, not your coin.
29:50Chris Romeo Now I have another slightly philosophical question, perhaps.
29:55Izar Tarandach Dun dun dun.
29:56Chris Romeo I thought all the new models that are coming out are supposed to have evil detection.
30:02Izar Tarandach We, we don't know which, which model is this, right? It could be.
30:07Chris Romeo Well, I mean, I can't think of another way to say it other than evil detection. So like, no, but it's, Have you looked at Mythos?
30:13Matt Coles I mean, Mythos will be evil if you ask it.
30:17Chris Romeo Well, there's new guardrails they've put around it though.
30:21Matt Coles Maybe it wasn't Mythos for the commercial.
30:22Chris Romeo 'Cause Mythos is still controlled to some degree, isn't it? Or is it not?
30:25Izar Tarandach But wait, there's enough people shouting around that guardrails are not enough.
30:29Matt Coles Ever.
30:29Izar Tarandach Never. Never. Ever. Ever never. Never never never ever never.
30:34Matt Coles More importantly, you can train your own model and ignore all the guardrails.
30:38Izar Tarandach Or you can take a model and just clean out the guardrails. There's tools for that.
30:42Chris Romeo Yeah, but not a commerc— not a Anthropic model or a public foundation model, an OpenAI model or something.
30:49Izar Tarandach That would probably be the wrong way of going for an operation like this.
30:55Chris Romeo Yeah.
30:55Matt Coles Also, those events would get logged.
30:58Chris Romeo Yeah. And you would lose plausible deniability, AKA you're illegal to think this is done by an open source model running locally.
31:06Matt Coles Probably.
31:06Izar Tarandach Probably. Well, locally somewhere, we don't know where.
31:09Chris Romeo Yeah, but I mean, not, it's not one of the big providers where there would be audit trails of what they did.
31:14Matt Coles Heck, this could be somebody running OpenClaw or NemoClaw or something like that, and they sent a Slack or WhatsApp message, hey, attack this site with malware, with ransomware.
31:24Izar Tarandach Make no mistakes.
31:25Matt Coles Make no mistakes. I forgot that.
31:27Chris Romeo That's the name of this episode.
31:30Matt Coles Make no mistakes.
31:32Chris Romeo Oops.
31:35Matt Coles Okay, well, don't, don't, don't act, don't act like an LLM. Be more human-like.
31:41Chris Romeo So, but I think if we, if we, if we start to summarize what we're seeing here, this is a new frontier that we're experiencing.
31:53Matt Coles It's only a new frontier in that it's something that humans could have always done but didn't because it was hard. And there's now, there's repeatable, it's weird, it's repeatable non-deterministic behavior, right? Which sounds like an oxymoron, but the fact that there's behavior that the LLM does, like insert default values because that's what it's been trained on. It goes back to, are they acting like a junior engineer? Or like a, you know, a high schooler, or are they more professional? And they're not yet professional.
32:35Izar Tarandach But it's not non-deterministic. At the end of the day, it's throwing stuff, getting the output, and acting on the output. That output is limited. The error message is not going to start screaming, unicorns are blue, and whatever, I don't know, and nice stars out tonight, and what are you doing this evening? It doesn't have to react to that. It has to react to path not found, right? And that's totally deterministic. The point is that now these things know how to, how to interpret those messages and perhaps without the plethora of ifs that would follow something like that in a, in a scripted attack. Go closer to the right one, right? Because they've been trained on that. But it's not like we, we, we are releasing masterminds that know how to adapt to every single situation in one shot and, and just go and boldly go where nobody has ever gone before and, and crack these things. That, that, that, that's not the situation.
33:47Chris Romeo Mm-hmm.
33:48Izar Tarandach When somebody No, I'm not going to say that, but when somebody is able to get a foothold for an agent into a target machine, and that thing has software that has not been seen before, there are no CVEs out there, there's no prior history of things like that. I mean, this is exploiting Postgres.
34:15Chris Romeo Well, it's an off-bypass for in whatever NACOS is from 2021.
34:20Izar Tarandach So can we say that these are things that it's been trained to recognize and it's recognizing them and using them, right? If it is an LLM.
34:29Chris Romeo Yeah.
34:30Izar Tarandach But when somebody drops an LLM in front of a machine that has nothing that has ever seen before and show me that thing doing recognition, getting persistence, figuring out what's happening around it in a completely, completely unknown environment, I'm going to be impressed as hell. Right now I'm just impressed. Show me that and I'm going to be very, very, very impressed.
35:01Matt Coles Right? And then you'll be scared because it will be coming after you.
35:04Izar Tarandach No, and then I'm going to leave computers forever and never touch anything electronic ever again. And I'm going to live in my backyard in a— And he's going to wear a tinfoil hat as as well. Well, wait, can you see it? No, I'm gonna move to a hobbit hole.
35:20Chris Romeo But, um, no, go west.
35:24Matt Coles Yeah, go west, young man. Oregon Trail.
35:29Chris Romeo Or you just die of dysentery again.
35:35Izar Tarandach So thing is, I think that it's coming. I think that it's great that people are able to do this research. I have huge respect for people doing this. And at the end of the day, I have to go back to Carl. Why didn't you patch it in 2025, Carl? So that it could be exploited in 2026. Why didn't you patch it in 2021?
36:05Chris Romeo Should have made an AI agent for patching.
36:09Izar Tarandach Uh, I think that there's a bunch around, but that's a very, very different discussion about gatekeeping in IT and people being able to say my machine is not working. Oh, it's because the AI patched it.
36:23Matt Coles So yeah, that's true. If you, if you patch this vulnerability, if you patch this 9.8 vulnerability, what do you lose? You lose internet access to your, to your Langflow. Yeah.
36:32Izar Tarandach Or what if the patch breaks back compatibility and now the thing that you are patching now dies because your patch—
36:42Matt Coles I'd say that's true. Sorry, you lose network access to your Langflow, right?
36:47Izar Tarandach Right. So it's not enough to be able to patch in place or to patch at AI speeds, to use the common parlance. You also have to be able to test that patch and make sure that you're not breaking other things because at the end of the day, the systems have to make money somehow.
37:06Chris Romeo Nobody has time to test the patch. Come on. You know it.
37:09Izar Tarandach They didn't have time before. Now they have even less time. So as long as that cycle keeps going, we are going to keep seeing those things.
37:18Chris Romeo I used to be in IT and they would say, you should test the patches. And I'd go, uh-huh, deploy.
37:24Izar Tarandach Yeah. What Friday evening? I mean, what could go wrong?
37:27Matt Coles What could go wrong?
37:27Chris Romeo I actually did, I actually did something really stupid on a Friday afternoon in trying to transfer a mail server. Oh, trying to transfer a mail infrastructure to Exchange. I ended up wiping out the Exchange server and had to rebuild it Friday night into Saturday morning and then restore it from tapes.
37:48Izar Tarandach So apart from the word Exchange in there, who hasn't?
37:51Chris Romeo Yeah.
37:51Matt Coles But restore from tapes, that, that's gonna get our, some of our viewers like, huh?
37:57Chris Romeo It wasn't a beta. It was not a Betamax tape.
38:01Izar Tarandach You, you are giving some, you're giving some, some, uh, some listeners, uh, and you're giving me PTSD.
38:07Chris Romeo But, uh, oh, myself.
38:09Matt Coles I'm, I'm bringing it back, making it real, physically real.
38:14Chris Romeo The, the real to real, baby.
38:16Matt Coles Real to real.
38:17Izar Tarandach Yeah, the, the fact of the matter is that I think that we are at a stage where it's time for people to question some of their past life choices and say, why didn't I patch it back then?
38:32Chris Romeo And the only life choices they're going to be—
38:35Matt Coles hindsight is 20/20 there.
38:36Izar Tarandach Exactly. And, and as we, we say in, uh, in Brazilian jiu-jitsu, there's only two great times to start. Which is when you're 6 or right now. So there are two great times to patch, when the CVE came out or now.
38:51Chris Romeo I thought you were going to say when you're 6. What am I going to do? How am I going to patch when I'm 6? I don't even understand computers.
38:56Izar Tarandach They are getting younger by the day, but that's because we are getting older.
39:00Matt Coles You can ask the computer to Eli 5 or Eli 6, right?
39:05Chris Romeo Dude, this is so— think about a kid right now. That's growing up right now, like they're, they're gonna, there's never gonna be a time that they, they're always gonna remember, think of AI as a thing. Like we have this multi-decade careers and history, you know, we remember all the way back to punch cards and they're not gonna gain, gain any of the experience that they get from talking to an AI. So that means we can work forever because we're going to dominate the industry.
39:42Izar Tarandach I remember when debugging AI meant mostly counting closing parentheses on Lisp programs. Okay, that's what debugging AI was back in the day. And thank God for Emacs, that closing parenthesis function was—
40:02Chris Romeo At least gave you some color to understand. How far you were off.
40:07Matt Coles Imagine if you had VS Code back then.
40:11Izar Tarandach Can I not?
40:12Matt Coles Can I not?
40:13Izar Tarandach Although, although it has to be said, the power of Emacs compels me. Back then, we already had scripts coming out from the internet that you could run and actually could wreck your whole thing. It's just that the people who wrote the scripts, dot Emacs scripts back then were not quite interested in wrecking your whole thing. People were a bit more—
40:37Matt Coles They were our age as we are now.
40:41Izar Tarandach As seniors from the '90s. It was a more elegant weapon for a more, for a more what? A more civilized time.
40:49Chris Romeo Right.
40:49Izar Tarandach So that was that. But I think that—
40:53Matt Coles They use their pinkies when they drink.
40:55Izar Tarandach Indeed.
40:56Chris Romeo Matt, you know you do.
40:56Izar Tarandach When they shoot you.
40:57Chris Romeo Come on.
40:57Izar Tarandach When they shoot you.
40:58Chris Romeo 4:00 PM every 4:00 PM. Matt's having tea time. See, that's what he does. Tea and crumpets.
41:07Izar Tarandach I think that the big takeaway from here is that it's coming. It's horrible. Let's all run in circles with our heads on fire.
41:18Chris Romeo But before we do that— Are we on fire or are our heads on fire? Are we supposed to light our head? Okay.
41:22Matt Coles He was going to say that, but he didn't want to scare our viewers.
41:27Izar Tarandach Now I've become conscious lately, so I know that I don't have a lot. I don't want to burn what I have. So, but, but, but at the end of the day, it's, it's patch people patch, right?
41:39Chris Romeo Patch.
41:40Izar Tarandach Damn it.
41:40Chris Romeo Wait, I feel like I've heard that before somewhere. Oh yeah. For the last 4 decades.
41:47Izar Tarandach Wait, can you say that one more time?
41:49Matt Coles So, so wait, we started with, instead of, instead of starting here, it Instead of starting a story about patching and ending up with AI, we started with AI and ended up with patching.
41:59Chris Romeo Patching.
42:00Matt Coles We're doomed.
42:01Chris Romeo Yeah. We are doomed.
42:02Izar Tarandach So I think that the title for this episode could be Cassandra, Make No Mistakes. Because again, we are Cassandra. We've been telling people for 40 years, if you don't patch this shit, it's going to burn. And hey, here we are.
42:20Matt Coles Well, with that, it was a Langflow. It'll be something else.
42:23Izar Tarandach Exactly.
42:24Chris Romeo I think that's a good place to end this happy, uh, happy, happy, joy, joy, as Ren& Stimpy would say.
42:31Matt Coles We're security people. We're never happy.
42:33Chris Romeo That's true.
42:35Matt Coles Unless we are the few, the proud, and the paranoid, though.
42:40Chris Romeo Okay.
42:41Matt Coles I was going through my old t-shirts, my old security t-shirts, and realizing that half of them are Probably no longer valid.
42:47Izar Tarandach This is the old—
42:49Chris Romeo these are the starters. My threat model is cooler than your S-bomb. Still true. Although— not cooler than your AI bomb, but—
42:58Izar Tarandach no, although it has to be said, soon we're going to have the TM bomb and this t-shirt will have to be, uh, offline.
43:08Chris Romeo I'll put it in the museum.
43:10Izar Tarandach Keep a, keep an eye on the, on CycloneDX. 'Cause the TM bomb is coming.
43:16Chris Romeo Yes. With that, we'll end this episode of Security Table. Thanks for tuning in. Keep your AIs safer and patch all your stuff is advice, partial advice we would have given you 30 years ago. We're going to repeat it again today. Patch everything.
43:34Matt Coles You can ask your AI.
43:36Chris Romeo What should I do? Ask your AI for the history of why you should have patched.
43:39Izar Tarandach Damn it, Carl.
43:42Chris Romeo Who is Carl?
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
