Is Spec-Driven Development Already Dead
With Chris Romeo, Izar Tarandach, and Matt Coles
Can a detailed specification make AI-generated software reliable, or does it simply move the ambiguity somewhere else? Chris Romeo, Izar Tarandach, and Matt Coles revisit spec-driven development and the promise that an agent can turn written intent into a correct implementation.
Part of: AI-written code and secure software development
Episode chapters · 13 chapters
- Intro: travel and movie reviewsAudio
- What is spec-driven development?Audio
- Repeatability and implementation variationAudio
- How do you verify generated code?Audio
- Test-driven development and weak AI testsAudio
- Engineering discipline and historical specsAudio
- Bounding a model for a specific ecosystemAudio
- Is spec-driven development already dead?Audio
- Ephemeral code and the Phoenix ideaAudio
- Patch the bug or regenerate the application?Audio
- Who is the better developer?Audio
- What you want versus what you asked forAudio
- Final thoughtsAudio
About this episode
Can a detailed specification make AI-generated software reliable, or does it simply move the ambiguity somewhere else? Chris Romeo, Izar Tarandach, and Matt Coles revisit spec-driven development and the promise that an agent can turn written intent into a correct implementation. They debate why earlier approaches struggled, whether natural language is enough, and how tests can fail when the same AI writes both the code and its checks. The conversation explores bounded models, the Phoenix idea of regenerating disposable code, and the choice between patching a defect and rebuilding from the specification. The security question remains: if an architectural flaw survives in the spec, what stops every new implementation from reproducing it?
Mentioned in this Episode:
➜ React
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 42 min conversation
0:09Chris Romeo Hey folks, welcome to the original, not to be confused with any cheap imitations, of the Security Table Podcast. My name is Chris Romeo, joined as always by Matthew Coles and Izar Tarandash. And they are elated, ecstatic to be here. They're literally jumping around in their video screens.
0:34Izar Tarandach Look, I survived to and fro with Air Serbia and, uh, it was an experience. I went to Belgrade and I'm back here to report.
0:47Chris Romeo Mm-hmm. You're not our, I guess you're not our traveling correspondent anymore then.
0:51Izar Tarandach Yeah. No, I went to Vienna by way of Belgrade and I'm never doing that again.
0:58Chris Romeo Belgrade. Where's Belgrade?
1:01Matt Coles He got Belgraded.
1:03Izar Tarandach Okay. So yeah.
1:04Chris Romeo Where is Belgrade? I'm honestly curious.
1:07Izar Tarandach It's 2 hours after Vienna. So you come from New York, you pass Vienna, you continue 2 hours and then you come back.
1:14Chris Romeo So I have an international travel policy. I never fly over my destination. While traveling?
1:21Izar Tarandach Let's just put it like this, uh, self-funding. So I said, you know what, let me save a night at the hotel.
1:30Matt Coles And that's the end of that right there.
1:31Izar Tarandach And yeah, let, let's stop there. It's not a place to—
1:33Chris Romeo So you slept in the Belgrade airport is what you're saying?
1:36Izar Tarandach No, no, no, no, no, no, no. But let, let, let's just say that, uh, the lounge was very Do you know that song Nikita? From like that you have like the overview of Checkpoint Charlie and all those like, that's the way it felt.
2:05Chris Romeo Took a while to get your bill.
2:06Izar Tarandach So you're saying I was afraid I was going to be taken into a trip on a ladder and lose my next connection or something. I've been watching too many movies.
2:19Chris Romeo Yeah, probably have.
2:20Matt Coles Yeah.
2:21Chris Romeo I'm still trying to figure out where Belgrade is. Never even heard of this place.
2:26Izar Tarandach Let's just put it like this. My seat, the audiovisual.
2:34Chris Romeo Yeah. Oh, I know where you are. Okay. Wow. That was the opportunistic path.
2:41Izar Tarandach Yeah. The audiovisual thing.
2:44Matt Coles AKA the cheap way.
2:45Izar Tarandach The audiovisual. Thing of the, the, the plane had an RJ45 connector slot in it.
2:53Matt Coles For an in, not an out.
2:54Izar Tarandach For an in. For an in. Yeah. So yeah, folks, glad to be here.
3:03Matt Coles He made it.
3:03Chris Romeo So yeah, I think you need a new travel agent.
3:06Matt Coles For those, for those who might be listening, for those who might be listening, if you're in Belgrade, please check in. We'd be curious.
3:14Izar Tarandach Yeah, I mean, and we love you.
3:15Chris Romeo I don't know why you didn't go through Kyrgyzstan.
3:19Izar Tarandach I couldn't. I don't think that any of my passports works there.
3:23Chris Romeo Okay. Gotcha.
3:26Izar Tarandach But, uh, no, to, to, to be honest, it, it was a very nice, uh, uh, trip and, and no, no words.
3:34Chris Romeo Okay.
3:35Izar Tarandach Words. The Air Serbia, the, the Air Serbia, uh, app. Could be vastly improved.
3:43Chris Romeo You can say that for almost every airline app though.
3:46Izar Tarandach There's really no, no, no, no. There, there are some, there are some that you can actually like until 30 seconds before you board the plane, you can still try and change your, your, your seat.
3:56Chris Romeo Yeah.
3:59Izar Tarandach Air Serbia says no seat for you.
4:07Chris Romeo All right?
4:08Izar Tarandach Did you have option— You fly me or you don't fly!
4:11Matt Coles Didn't do any security analysis on the app too though.
4:14Izar Tarandach Oh no, no, no, no, I was very well behaved traveler. Very well behave.
4:22Chris Romeo How long was flight path? Just curious now over 30 hours.
4:28Izar Tarandach Let we see uh, 2001 Odyssey In Space, ah... The Hobbit, The Desolation of Smaug, something like 11 hours measured in, in movies.
4:39Matt Coles Is that what you're saying?
4:41Izar Tarandach Something like 11 hours. Yeah.
4:43Chris Romeo Interesting view into the mind of Isar here by the movie choices.
4:47Matt Coles Or was that what was the new ones?
4:50Chris Romeo What's the newest? 2001: A Space Odyssey, new release.
4:57Izar Tarandach I was basically entertained by something as old as me.
5:01Chris Romeo Well, I was recently on an airplane as well, on, and the airline that, uh, I was flying on, I was able to watch Hail Mary, new sci-fi.
5:09Izar Tarandach Mm-hmm.
5:09Chris Romeo Yeah. Pretty good. I read the book. The book is obviously better, which is always the case for everything, but still, well, pretty well done. Only fell asleep for about 30 minutes in the middle of it.
5:22Izar Tarandach So it's, it's, it's worth it. Have a lot of popcorn at hand. It's, it's worth it.
5:28Chris Romeo Yeah.
5:29Izar Tarandach Very different from The Martian, but, uh, like The Martian, you have this illusion that it could actually happen.
5:35Chris Romeo Yeah.
5:35Izar Tarandach The other one, it's kind of like, so it's like Interstellar then? No, because nobody's trying to explain physics to you. But, uh, it's, it's, it's very easy to poke holes at, and it's very easy to not care about the fact that you are poking holes at. It's Interstellar with the Disney treatment, I think.
5:57Chris Romeo Interesting.
5:58Matt Coles Okay.
5:59Izar Tarandach Like you finish, you finish the movie thinking, where can I get a plush toy of the alien?
6:08Matt Coles Okay. So true. So, so kind of like Contact.
6:13Izar Tarandach Wait, Contact is the one with— Jodie Foster.
6:15Matt Coles Jodie Foster.
6:16Chris Romeo Oh yeah.
6:18Izar Tarandach Definitely. Totally.
6:21Chris Romeo Okay.
6:21Izar Tarandach Oh, off the Alien.
6:22Chris Romeo No, we're going to, we're going to wrap up the movie review segment of the Security Table and move on to the spec-driven development segment.
6:32Izar Tarandach Yeah.
6:32Chris Romeo Which that, I mean, if we don't win an award for that transition that I just did there, there should be an award for that because, you know, look, it's a movie in itself, right?
6:44Matt Coles Independence Day could have been one aspect of development alone.
6:48Chris Romeo You said Independence Day?
6:50Matt Coles I did.
6:51Chris Romeo Dude, don't even tell me this.
6:53Izar Tarandach Which I could have watched actually, but it was too short for my trip.
7:00Matt Coles At least it wasn't Martian Attacks.
7:02Chris Romeo Mars Attacks.
7:04Matt Coles Oh, Mars Attacks.
7:04Chris Romeo You're right. That's classic. That's a classic. That is so good. Yeah, we need to— And I went hoping for Minions and Monsters. Those were invented far too many decades after your plane was constructed.
7:23Izar Tarandach Yeah.
7:24Chris Romeo Okay, Matt, what is spec-driven development? I feel like I've been living under a rock and I don't know what this means.
7:32Matt Coles I'm going to probably piss off a lot of people by saying this, but it's bringing back software development to the old days when we used to write specs and then code to it.
7:43Izar Tarandach Oh boy, boo!
7:44Matt Coles It's the newfangled software engineering paradigm where you write a specification and then you get the AI to implement it for you as opposed to writing a specification and getting the human to implement it for you.
7:58Chris Romeo So wait, isn't that what people are supposed to be doing now? Like what are, or are people, I'm very confused. Are people just going to their Claude prompt and saying, do my work for me? And I'll see you after lunch.
8:11Matt Coles Pretty much.
8:13Izar Tarandach No mistakes.
8:14Matt Coles No mistakes. Yes.
8:15Chris Romeo You gotta specify no mistakes.
8:17Izar Tarandach No mistakes.
8:18Matt Coles We need to, we need to find that cartoon that somebody put out. I think they put it out on Reddit about, we're gonna be able to, with Spectrum Development, we're gonna be able to create this description of the system that we want in perfect detail, and then we're gonna get a perfect representation of it. We used to call that code.
8:34Chris Romeo Yeah, exactly.
8:36Matt Coles I'm sure I didn't do it justice, but that's what he was saying.
8:39Chris Romeo There's no world where code was ever a perfect representation.
8:43Matt Coles No, no. And there's so many ways of representing things wrong.
8:47Izar Tarandach Code was always the perfect representation. The thing is that the code that you got to might not represent the thing you wanted it to represent. But code was always the perfect representation.
8:59Chris Romeo Words mean things. The word "perfect" means what?
9:02Izar Tarandach Okay, without flaw—
9:04Chris Romeo Whatever is running.
9:07Izar Tarandach Okay. Whatever is running is running because there is code saying this thing runs this way.
9:14Matt Coles Right.
9:14Izar Tarandach So the code was always the perfect representation of whatever was running.
9:18Chris Romeo Yeah.
9:19Izar Tarandach Yeah.
9:19Chris Romeo I see where you're going.
9:20Izar Tarandach But in your head, you have a system that has its own perfect representation in your head. And what was always missing was the bridge between the perfect in your head, the Platonic idea of the shape of a thing and the code that ended up being the perfect representation of whatever is running.
9:42Chris Romeo Gotcha. So yeah, okay. I'm with you now.
9:44Izar Tarandach Now to bridge back with our, with our, with the way that we moved from one thing to another. I think that this spec-driven thing is basically somebody wrote a book, somebody read, read the book, wrote a script of the movie, and people are watching the movie and saying, oh, this doesn't look like the book. Because the book always has more details, the book is always more involved, the book is always deeper, and the scripted presentation is always, we got 2 hours and we shouldn't be boring people, so let's put some explosions in there.
10:19Matt Coles I mean, so Spec-Driven Development basically is you tell, you describe what you want, and there's varying degrees of specificity, of course, right? You could say, I want a front end to take input and do something with it and make output. Or I want a system that is written in Python and hosted by a React script using this library and that library, and it takes input and it does kind of canalization and input validation and some other stuff, and then it does some math in the backend and then it writes to disk. And expand on those for detail. And the AI, you ask the AI to interpret that in a way that of course AI interprets that and does that iteratively and then generates a set of tasks to do and goes off and spins off that work. And eventually you end up with a thing, hopefully.
11:24Izar Tarandach So basically it is, let's take vibe coding, Let's take the vibe out of it, and we are left with coding.
11:35Matt Coles It makes every developer an architect. Actually, not even, uh, not, not quite an architect, right? Because they're in part an architect, but it also in part, I don't want to say a technical writer because technical writers do something different, but it makes them write down what they intend to do before they do it.
11:58Izar Tarandach It sounds like it is the embodiment of English is the new coding language.
12:06Matt Coles Yeah, that's, that's, that's the thing. I, I don't, I don't understand that at all because people have been using non— people can use non-English for AI code, for AI work, because AI understands all of the languages. You're not bound to English.
12:21Izar Tarandach Yeah, but I think that they're just placeholder of English 'cause—
12:26Matt Coles So language is the new programming language?
12:30Izar Tarandach Yeah, exactly. Natural language is the new programming language.
12:33Chris Romeo Great, I gotta learn another language? I gotta be able to speak another language?
12:36Matt Coles You have to be able to speak naturally.
12:38Izar Tarandach Hey, my son is already doing dual language Chinese. No judgment here.
12:42Matt Coles You don't even have to understand it to do it apparently.
12:44Izar Tarandach True.
12:46Chris Romeo So I'm still trying to understand, like, what do I get out of this thing?
12:51Matt Coles You don't get—
12:51Chris Romeo why am I wasting my time? It's because it's not repeatable, right? It is. It is.
12:57Izar Tarandach It tries to.
12:59Chris Romeo Yeah, but LLMs aren't repeatable. So I can't get—
13:01Matt Coles that's true.
13:02Chris Romeo I'm going to get a different result even though I start with the same source document. It's never going to give me the same thing 2 times in a row though, right? Correct.
13:09Matt Coles So if you write a spec, you'll get an infinite series of possible outcomes. By the way, you know what that used to look like? You had human developers with a spec and infinite possible outcomes.
13:22Izar Tarandach I'm going to have to push back on that one.
13:25Matt Coles Okay, please do, because I would like to know if you have two developers who automatically code the same.
13:30Izar Tarandach You are getting the possibility of an infinite amount of outcomes, but because the outcome is bounded to a programming language and you only have so many variations that are valid syntax, and the AI has been trained on valid syntax, then you're taking an infinite and you're applying a very—
13:54Matt Coles I was saying infinitely metaphorically here because in the broader sense, right? Because if you have 10,000 developers, you'll get 10,000 implementations.
14:03Izar Tarandach If you're being metaphorical, then yeah, definitely. But it's just that people have been banding the word infinite lately. Around—
14:09Chris Romeo sorry, when you say—
14:10Matt Coles I'll be better about my, my boundaries.
14:13Chris Romeo Yeah. When you say infinite, there are 10% on either side of the solution that developers have come up with, which are just ludicrous. But yet an LLM may create a ludicrous solution. Oh yeah.
14:27Izar Tarandach 'Cause my next segment—
14:28Chris Romeo it may recreate all the libraries that it thinks it needs. It may create 5 times the code it needs to sort something.
14:36Matt Coles You may tell it to use Python and it'll go off and implement in Ada because it decides otherwise.
14:42Chris Romeo Yeah.
14:42Izar Tarandach And do not forget that just because it's ludicrous does not mean it hasn't been released.
14:49Chris Romeo True.
14:50Matt Coles And on top of that, so the other, I guess the other paradigm here is not just you do spec-driven development, but then you tack onto it, you know, how are you gonna verify that what comes out is correct. So you need a correctness check. So you have to implement some sort of testing process as well, right? Because you just have— if you just do speculative development, you're not guaranteed to get something functioning. You're not guaranteed to get something that matches or aligns to your spec. And now you may not get anything that's correct.
15:24Chris Romeo I need DAST to check it, what you're saying.
15:27Matt Coles AI is the new DAST. That's what you're saying, to check itself.
15:31Chris Romeo Keep sending in. Okay. So, so now you're introducing some new piece here though, Matt. You're saying that spec-driven development creates some code and we need something to check the output of the spec-driven development. Is that— and I hate to be the category of things that we've had in the past, SAST, DAST, IS-RASP, SCA, functional tests, unit tests.
15:56Matt Coles I mean, we're not talking necessarily security here. Because the spec is not necessarily just a security spec. In fact, if you're lucky, there's a security spec that goes along with it, right?
16:08Chris Romeo Do you—
16:08Matt Coles or the security as part of the spec, because—
16:10Chris Romeo do you employ test-driven development in conjunction with spec-driven development to have it create your tests first?
16:17Matt Coles It seems like that's an ideal case. That's a better case than having a test plan implemented after the fact.
16:25Izar Tarandach But how many times have you guys seen AIs writing tests that it's basically our return true.
16:32Chris Romeo And that's how I write tests too.
16:35Izar Tarandach And then it comes and says, I tried everything. And you say, did you? Did you really? Uh, no, you were right.
16:40Matt Coles Oh no, you're right. I, yeah, I missed that.
16:44Chris Romeo Of course, just like a human.
16:47Izar Tarandach No, the worst is I'm so happy you pushed back on that. Oh, are you? Cool.
16:51Chris Romeo Yeah. Why didn't you just do it right the first time? Yeah.
16:54Izar Tarandach No mistakes.
16:56Matt Coles So, and I hate to be that naysayer of, you know, everything AI is new, but really spec-driven development is just bringing us back to old spec-driven development.
17:07Izar Tarandach Good times.
17:09Matt Coles And it's not necessarily a bad thing because of rapid iteration, right? You can say, I have the spec and I write, it creates something and then, oh, that's not right. And I update the spec.
17:18Chris Romeo Yeah, but the challenge in previous—
17:19Matt Coles And then redo it.
17:21Chris Romeo Spec-driven development is that people didn't really follow the spec. They wrote the spec because they had to have it. I'm just thinking about my history in large corporate environments. You had to have a number of spec documents as part of the product lifecycle because you had to follow them.
17:39Matt Coles There was nothing that said, well, you, you were supposed to have, that was the whole like thing about doing ISO, right? Was you have specs that match what you actually write. So you get, you get, you get, if you define it up front and you follow the spec, you get what you end up at the end. And you can do formal verification and all that sort of stuff.
17:55Chris Romeo Nobody ever does that.
17:55Matt Coles And it never worked. So what we're really doing here is we're bringing it back to 1950s engineering process with AI.
18:05Izar Tarandach Yes, but—
18:07Matt Coles Hey, we've hit the pinnacle. If that's true, we've hit the pinnacle. But we as engineers, we as engineers define what we want in a precise way, fairly precise way. But that's the thing. We get the LLM to almost implement that in a precise way, and we can verify that. Well, I would say without the almost, but of course, because it has variability, it's no guarantee. But we're getting better because we can iterate on it faster. So really, we've met the pinnacle of engineering practice.
18:39Chris Romeo All right, I'm retiring then.
18:40Izar Tarandach What do we have?
18:41Chris Romeo Oh, whoa, I'm back.
18:45Izar Tarandach Remember those times that we We went to, to try to model something and we got the Eureka moment where the architect would, would, would tell us about the spec and how the thing works. And then the junior developer would raise its hand and say, ah, you know what? When we were implementing the thing, we found out that the spec didn't really know what it was talking about because the things that it asked us to implement actually work in a completely different way. So we had to sort of like. The, it accepts the input, it puts out the output, but inside it's completely different. I mean, we all had that experience repeated.
19:21Matt Coles So this is, so what you're saying is we've met the, we've met the potential of 1950s engineering practice with all of the problems of the engineering practice.
19:31Chris Romeo Well, not really though, because the AI is gonna, it's going to attempt to follow what the spec says.
19:37Matt Coles Right.
19:38Chris Romeo Or it should tell you, it should give you some feedback to say, this is complete nonsense. This can't be.
19:44Matt Coles It won't though. It won't because it wants to appease you. So it's going to try and follow the spec, you know, verbatim.
19:50Izar Tarandach It will gladly build a car with the wheels on the roof. But before we actually, we actually, uh, kick it to the curb, I, I want to remind people that what was done in the '50s took, took a bunch of men to the moon and back. Including some very bad things that happened one way or another.
20:19Matt Coles So because the specs couldn't account for everything.
20:21Izar Tarandach Yeah. But something there worked. Perhaps it didn't work at the cloud speed, but right.
20:31Matt Coles That was the, that's the key part, right? It worked at engineering speed where you had rigid, where you had. I mean, if you— aspect-driven development may not work for hardware engineering because of the rigidness and the—
20:48Izar Tarandach the rigidness of what?
20:50Matt Coles We, we, things worked because people spent time refining the specification, doing the R&D and being rigid about it. Did what I do implement the spec and does the spec address the concerns that I have for when I put this into production. Whereas somewhere along the way, like in the '80s, we like make it work and ship it.
21:14Izar Tarandach But wait, wait, wait, wait, wait. Wouldn't—
21:17Matt Coles sometimes it was just ship it. Don't even make it work. Just ship it.
21:20Izar Tarandach Right. But wouldn't being rigid make it, sorry, easier for an AI to, to implement it because it's giving more guardrails?
21:31Matt Coles You mean implement, have a coding convention and a standard and have it only implemented that?
21:38Izar Tarandach That I found that it doesn't always work that way. But what about having the wiggle space, which I'm using here as a technical term, the wiggle space.
21:53Matt Coles Wiggle space, very technical.
21:58Izar Tarandach I'm going to call it the movement variation field, aka the wiggle space, uh, that the AI would have to implement that spec, right? Assuming that I'm not talking about some generic AI, but assuming that you get to a model that you actually get trained with bits and pieces of code that are restricted in the same way. So it doesn't know about all possible Python libraries out there, but it knows that if it has to put two numbers together, then there's only two ways that you can write a equals b plus c or a equals c plus b. So, well, of course you can do all kinds of other tricks, but let's stick to that. Wouldn't making the model limited to the universe that that spec knows about. Create a pipeline that leads from spec to more provable and reliable code.
23:06Matt Coles Maybe, but let me ask you this. At what point does it make it— what, at what point would it be just easier to have a code generator where you define the model and have it generate code deterministically against that model precisely each time?
23:23Izar Tarandach That's a damn good question. Because we know that for hardware, this is for chips, right? At some point they moved to that approach. You had a very descriptive language, but very bounded, very rigorous.
23:37Matt Coles Do you remember UML?
23:39Izar Tarandach Oh my God. I try to forget it every day.
23:42Matt Coles But you remember what UML was supposed to have done?
23:45Chris Romeo No.
23:45Izar Tarandach Databases, right?
23:47Matt Coles Well, uh, no, classes, classes, classes.
23:50Izar Tarandach Yeah, yeah, yeah, yeah.
23:51Matt Coles And, and code generation, right?
23:54Izar Tarandach Yes. Oh my God. I had forgotten all about that. Thank God. But now I remember and now I have to forget it again.
24:00Matt Coles So, so, but at some point, if you train a model and refine it to bound its outcomes, at some point you're going to reach a, reach a well, whatever, on a curve, I don't know, whatever that is, where plateau or whatever, where you, where it may just be easier to do one of these model generation things deterministically, like we, like was promised back in the, in the '80s. You would think. A lot of crap came out of the '80s. I mean, come on, don't be making fun of the '80s.
24:30Izar Tarandach The '80s were good. The '80s were fun. I remember the '80s fondly.
24:36Chris Romeo So, Izar, back to your example though, Early in your, that, that previous example, you were talking about what sounded like a custom model for that particular spec. Isn't that just, doesn't that just throw the idea completely out the window? Because how are you going to generate a custom model specific to each code spec you're trying to generate?
24:59Izar Tarandach No, no, not to each code spec, not to each code spec, but chances are that people using those specs live in a certain.
25:07Chris Romeo Ecosystem.
25:08Izar Tarandach Ecosystem.
25:10Matt Coles And so you have a spec, you want a spec to generate a React front end. You may only train it on web security and how, and user experience and interface design and React JavaScript.
25:23Izar Tarandach Yeah.
25:24Chris Romeo And then you bound it somehow with a guardrail that says don't create anything other than a React. We don't create anything else.
25:31Matt Coles That's the guardrail. The model can't because it only knows those topics.
25:34Chris Romeo I see.
25:35Izar Tarandach Yeah.
25:35Matt Coles You're limiting its knowledge.
25:36Chris Romeo But isn't that, isn't that computationally expensive to create that mo— that custom model and update it and maintain it.
25:44Izar Tarandach It's actually, I don't think, I think the creation is the creation of the corpus of data that you would use to train that.
25:50Matt Coles And in fact, I think there are techniques, I think there are techniques that are reliable for taking models and removing content from them. So in theory, you could take a public model, like a, you know, a 20B public model and remove everything that you don't need. But in doing so, is it really efficient to do that? I mean, I guess if you're gonna create, that's what I'm asking. If you're gonna, if you're gonna create a number of these interfaces, but if you're doing just one, is, is it, is it worth it?
26:23Chris Romeo I mean, who does anything at the scale of one though?
26:26Izar Tarandach No, it's like doing a lobotomy. It's, it's like doing a lobotomy just to get a person to be able to do one task.
26:33Chris Romeo Right.
26:34Matt Coles So that would be your web front end, you know, model. And then you'd have a mid-tier model and you'd have a database layer model.
26:40Chris Romeo And then you'd have a— Hey, idea for a new company. Name: Lobotomy.
26:43Matt Coles Oh, Lobotomy.
26:44Izar Tarandach Yeah.
26:44Chris Romeo Lobotomy.com. Love it.
26:48Izar Tarandach Lobotomizerus.
26:51Chris Romeo We'll take a model and we'll strip it down to make it only do one thing.
26:56Izar Tarandach Yeah, come with a brain labeled as back.
27:00Chris Romeo That's good marketing right there.
27:02Izar Tarandach Right?
27:03Matt Coles You heard it here first, folks.
27:04Chris Romeo Uh-huh. Trademark patent pending. Yeah.
27:09Izar Tarandach And paper bunny is— but anyway.
27:14Matt Coles Plushies coming soon.
27:15Izar Tarandach Plushies coming soon, exactly, of Jodie Foster.
27:18Chris Romeo So are you prescribing a world where people use this? Forever, like, let's just say you have a mid-sized company, you've got 250 developers on staff. Is this something that you try to roll out across the entire engineering organization?
27:34Izar Tarandach Or like, what do we do with this thing at the end of the day? But let's just open a parenthesis for a second. There is a growing body of people who are already saying that Spec-Driven Development is dead.
27:47Chris Romeo Yeah. Great. Something else is dead. Everything's dead.
27:50Izar Tarandach It's a non-zero number of people who are saying that.
27:55Matt Coles Yep. It's something, there's fact-based, it's write facts, don't write specs, write facts. No, that's not it. That's a different thing. Context engineering is dead also, I think, right? So there's, AI is moving so fast that it's impossible to know these days.
28:13Chris Romeo So basically everything's dead. How do we know it's alive? Is there a—
28:18Matt Coles It's Schrödinger.
28:20Izar Tarandach It's dead and not dead at the same time.
28:22Chris Romeo That's an interesting idea.
28:27Izar Tarandach But actually going down on the article here, we get to spec-driven development was really hard to communicate to regular developers. What Chad did really well is ground the story in code. Saying that the only difference is that this code is no longer permanent. It's ephemeral and it can always be regenerated from other artifacts. To say that I'm completely lost.
28:56Matt Coles I mean, I mean, so, I mean, all right, let's think about this now. Bring, bring it back to the, to the classical engineering approach of you have Spectre in development, you have a spec, you have a spec. That drives development, Spectrum development, and then you find a bug. What do you do?
29:19Chris Romeo So it's a regen.
29:20Matt Coles Well, that's the question. So do you—
29:22Izar Tarandach Open a ticket.
29:23Matt Coles Do you patch the bug or do you reconstruct the application and hope that it doesn't have the bug? Now, I, it's an interesting question. If you can write the code fast, if you can write code, bespoke code faster than finding and fixing a bug, which of course may be an architectural flaw, right? And by the way, if it is an architectural flaw, you really need to update the spec because the spec was wrong or imprecise, or there weren't enough guardrails with that spec to implement a proper architecture. But if you have a bug, Is it more efficient to just regenerate the code or to patch the bug?
30:09Izar Tarandach But wait, if you patch the bug, you're going to have to regenerate. No, wait, regenerate the code from scratch.
30:14Matt Coles Rewrite the code from scratch.
30:15Chris Romeo From scratch. Have the AI just try again, but then it's going to create a different bug.
30:19Matt Coles It's going to, right, exactly. Right.
30:22Izar Tarandach So why do we let people like this close to computers? Why?
30:26Matt Coles Why?
30:26Chris Romeo Why?
30:27Matt Coles Because you have people who don't have computer science degree.
30:29Izar Tarandach Why?
30:29Matt Coles Because of the democratization of computer science now, right? So this is the whole notion of AI now allows people who don't have engineering backgrounds to do engineering work. It's like having them do security work when they had DevSecOps. Remember, we gave engineers the authority and responsibility without the capability to do security.
30:51Izar Tarandach Hey, here's a scalpel, go do neurosurgery.
30:56Chris Romeo So you got YouTube these days, you can do neurosurgery, just watch a YouTube video. Yeah.
31:01Matt Coles And stay in a holiday.
31:01Izar Tarandach Yeah, but only once.
31:05Matt Coles Well, your patient will only need it once. Yeah.
31:08Izar Tarandach I suggest that people try it on themselves first.
31:10Chris Romeo That's true. And that's lobotomy.com. Try that yourself first. All right. So are we in favor of spec-driven development? Are we against it? Or do we just not know?
31:20Matt Coles I, it depends if you consider what's he— what are, what, what should you do with all those engineers out there? What is their role? Is their role writing specs?
31:32Chris Romeo I think so.
31:34Matt Coles Because that's what we're talking about here, right? Developers write specs.
31:37Chris Romeo I think so. I think from what I've been talking to some engineering folks who are using AI, it's, I will say it's scale right now. Like they're really using AI to generate code. This is the model that I've heard that high-efficient organizations are going to is really focusing on documenting the spec correctly and then letting the AI agents do their magic.
31:59Matt Coles Yeah.
32:00Chris Romeo Versus trying to marry the two together in a collaborative approach.
32:05Matt Coles Right. So question, does anyone remember during the Agile craze of the 2000s, I suppose, when somebody, I don't know who it was, said the code is the documentation.
32:21Chris Romeo Oh God.
32:24Matt Coles And now you have developers, those same developers, probably those same engineers saying, hey, I'm gonna write documentation for a living.
32:32Izar Tarandach I was in the team that said that if it was hard to write, it should be hard to read.
32:38Matt Coles Well, in this case, it would be impossible. In this case, it would be impossible to write because you're not writing it.
32:43Chris Romeo So look, yeah. So I guess that, that, that brand, I'm gonna, I'm gonna even, I'm gonna, I'm gonna raise you 25 philosophical points. And say, who's a better developer, AI agent or human developer? Because that answers your ultimate question. If the AI agent is better at taking the spec and turning it into functioning code and creates less bugs and less security vulns than the human does, then yes, the engineer becomes a spec document writer. If not, maybe it's the opposite way around, but that's the philosophical question we have to figure out.
33:18Matt Coles Well, and I think the answer is an interesting, it's an interesting answer. How many times, how many iterations does it take the human developer to be perfect or precise enough compared to the AI who can just spit out an infinite series?
33:35Chris Romeo You, you're using the word perfect again, so I'm gonna say infinite to both answers.
33:39Matt Coles So, uh, let me, let me, I, I, I used, I used infinite before incorrectly. I used perfect incorrect, incorrectly now sufficiently correct.
33:49Izar Tarandach I am going to push back on both. I'm going to say that first question here is, which one of them is better at giving you what you want, not what you asked for? Back in the ice ages, that means 4 months ago. You would give a, air quotes, spec to an AI and it would gladly go and build that car with the wheels on the roof.
34:23Matt Coles Mm-hmm.
34:24Izar Tarandach Right. Nowadays, it's going to stop and say, wait, I see that the wheels are on the roof. Are we building an inverted car? Would you like to, uh, invert the wheel as well? Uh, do you have the right idea? Do you want to ask, uh, somebody before we do this? So the, the AI is, is offering a level of intuition of what the stupid thing that you asked for actually needs to be that wasn't there in the beginning. And developers always did this to certain degrees of pushback.
35:05Matt Coles So you had spec, you had spec reviews and those spec reviews, you would reveal I, because it was, but at that time, remember the specs were not written by developers. They were written by product owners or product managers, or they were written by architects who in theory were completely removed from the actual engineering function. Right. And, and so you want us to build this self-floating, self-driving thing that looks like a hoverboard.
35:37Chris Romeo Sure.
35:38Matt Coles Do you have advanced physics I can use?
35:40Izar Tarandach So that's the thing that the developer always, mostly always brought in the WTF filter. They were able to read what was being asked and say, this is not how these things work.
35:54Matt Coles Or I could build this once and then, right?
35:58Izar Tarandach The AI is only beginning to bring the thing in.
36:01Chris Romeo So who's better? The AI agent or the human developer?
36:05Matt Coles Describe better. What's your, what's your measure of better?
36:08Chris Romeo Uh, sufficiently correct. So again, creating sufficiently correct output.
36:11Izar Tarandach So again, I, I, I go back to my point. What, what is better? Something that does what you asked for or something that does what you wanted?
36:20Chris Romeo It should, uh, it should understand what I want based on what I ask.
36:23Izar Tarandach Then I would say it's still a person, but the AI's coming up with some very interesting ways of asking you Hey, is this what you actually want?
36:33Chris Romeo So I got good news for all of us. Uh, we're now qualified as AI philosophers so we can apply at Anthropic and make like $2.5 million a year as AI philosophers.
36:45Matt Coles Yes.
36:50Izar Tarandach But, uh, the, the, just to, to go back to this, uh, uh, to this article here. We didn't touch on two things. First of all, the whole offer here is an open-source specification language that uses structured human-readable documents as the single source of truth for how software should be built and behave.
37:16Matt Coles That's, uh, we're talking about, uh, something from GitHub by any chance?
37:21Izar Tarandach Uh, no, something about the person who wrote the style. And so it's not only that you're going to write the spec, but hey, there's a spec to how you write the spec, right?
37:35Matt Coles So everyone has conventions.
37:38Izar Tarandach Yeah, but those conventions, they are in there so that things become repeatable and things become parsable. So it's not just about sitting down and writing a spec. There's just a method to the madness.
37:52Matt Coles It always should have been a method to madness though, right? Yeah.
37:57Izar Tarandach The thing that really bothers me about this whole thing is this whole discussion of fix the bug or regenerate the whole thing. And they use the analogy of the Phoenix, right? The code is like Phoenix. It gets regenerated every time that it's needed. So you don't have to worry about the code. You only have to worry about the spec.
38:18Matt Coles The only downside to that is the Phoenix is a clone.
38:23Izar Tarandach That, that, no, that, that's the thing. It's not the only, it's not the only one.
38:27Chris Romeo It's not a perfect analogy.
38:28Izar Tarandach No, no, no.
38:29Chris Romeo It's not a perfect analogy. But I mean, one of the examples that I'm just reading here.
38:32Matt Coles It's not a good analogy.
38:33Chris Romeo One of, but listen to one of the examples they give.
38:35Izar Tarandach It's a very bad analogy.
38:36Chris Romeo This is, this is a really powerful use case for what, for this technology. Um, using Codeplane to build and maintain integrations with external data providers. When an external API changes and breaks an integration. Codeplane can often fix it by regenerating from the same unchanged spec because the spec encoded intent on implementation, meaning it's reading an API, it, it'll go read the external API spec and then adjust and then build a new version of the integration that understands whatever they changed on the API. So like, that's, that's a, that's an obvious win for this type of tech.
39:10Izar Tarandach No.
39:10Matt Coles And, but, and by the way, that required, that requires modular development. You need a modular architecture to be able to plug in new APIs without reconstructing the whole rest of the system. So there's a developer, so there's an architecture paradigm that has to have been followed originally.
39:26Chris Romeo These are saying no to this. I'm curious.
39:28Izar Tarandach Did you just invent a way for my system to be influenced by the bugs of other systems? Somebody created an API that doesn't have authorization authentication. All of a sudden, my system is going to wake up one day, look at it, say, Oh, look, the API changed. I should recompile myself and forget about that authorization authentication thing because the API doesn't use it.
39:48Matt Coles Pretty much.
39:48Chris Romeo Yeah. I mean, security doesn't really exist in an AI-generated world.
39:52Matt Coles So unless you intend it, he's about to explode.
39:58Izar Tarandach I am very bothered. I am very bothered. I may need a plush animal of an alien. Yeah.
40:05Chris Romeo All right. Well, we're just about out of time here.
40:07Izar Tarandach Any final thoughts on, uh, Meteor 2026, because clearly we can't go anywhere else.
40:17Chris Romeo Oh, coming all the way back around. I mean, the movie Hail Mary.
40:21Izar Tarandach Crap, man.
40:21Chris Romeo It was about solving a problem like a meteor.
40:24Izar Tarandach Let's not try to save it. Let's just let things take their course.
40:29Chris Romeo I mean, that's one outcome for humanity.
40:31Izar Tarandach We got there. It's like we deserve it. I mean, we asking for it. Come on.
40:41Matt Coles All right, now we, now we can't apply to Anthropic.
40:46Chris Romeo He just got disqualified.
40:48Izar Tarandach OpenAI, here we are.
40:50Chris Romeo Call us.
40:51Matt Coles No, it'd be my—
40:54Chris Romeo I think that's enough madcap fun philosophy something else. I can't even describe what just happened here.
41:02Matt Coles But she's en route.
41:06Chris Romeo Yeah, it was, it was a good journey that went somewhere. And if somebody's listening could figure out where that was, just shoot us a nice email.
41:13Matt Coles So we went from Hail— we went from Project Hail Mary to Interstellar, in truth.
41:16Chris Romeo That's a good title. It's good. We should make our own movie called that, Hail Mary to Interstellar. All right, well, tune in again soon for another example of madcap funnery, philosophical hijinks. And breaking social constructs. I don't know, I'm out of ideas. Thanks for listening, folks.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
