A Convergence of AI in the World of Cybersecurity
With Chris Romeo, Izar Tarandach, and Matt Coles
Izar, Matt, and Chris scour the Interwebs for an article to discuss, only to find that each person has chosen an article related to the convergence of AI and cybersecurity. We discuss whether ChatGPT can replace humans with threat modeling, Microsoft's Security Copilot, and the open letter to freeze AI development for six months.
About this episode
Izar, Matt, and Chris scour the Interwebs for an article to discuss, only to find that each person has chosen an article related to the convergence of AI and cybersecurity. We discuss whether ChatGPT can replace humans with threat modeling, Microsoft's Security Copilot, and the open letter to freeze AI development for six months. AI is the future, and it will significantly impact the security professional's role.
Follow AI Security Table:
➜ Home: https://securitytable.ai/
➜ X: https://x.com/AISecurityTable
➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/
➜ YouTube: https://www.youtube.com/@AISecurityTable
Transcript · 46 min conversation
0:13Chris Romeo Hey folks, welcome to another episode of The Security Table. I am joined by Matt Coles and Izar Tarandash. My name's Chris Romeo, and we have decided that this episode is called— I don't know, Izar had some fancy name for it, the convergence of AI or something. Because we thought, hey, let's take a look at the news, see if there's some things we can talk about. We'll all bring an article. Guess what? We all brought an AI-influenced article. So, and none of us typed into ChatGPT, what article should we bring to the security table? Like, we all used our own brain power to—
0:48Izar Tarandach Oh, sure.
0:51Chris Romeo Okay. Well, one of us was, the AI was taking, giving some influence on the direction that we go. But let me introduce the first article, the one that I found that I— it's from a few, I don't know, a few months ago, I think. Yeah, back in December, our good friend Adam Shostak did an article for Dark Reading, "Threat Modeling in the Age of OpenAI's Chatbot." And so kind of an interesting take on threat modeling in that, will ChatGPT cause threat modeling to not be so human-driven? Could threat modeling replace us as threat modelers? And so Adam went through and kind of did a great job explaining a little bit about GPT, talked about something that we'd seen on the OWASP Slack where somebody was using ChatGPT to do, you know, to put together a threat model. And then, you know, he offered some other thoughts about it. But I thought this was an interesting take given the, the depth that all of us spend thinking about threat modeling, you guys writing about threat modeling. So let's just kick it off from there. Like, what do you think? Let's start with Matt. Like, what are your— what do you think about this idea of ChatGPT influence threat modeling?
2:05Matt Coles Well, I have a feeling that ChatGPT is just the tip of the iceberg here, right? And that as GPT-4 and Bard and these other, other technologies come online, we're going to see them get more and more better, more better, better in terms of synthesizing information, which ultimately is what we as security practitioners do when we do threat modeling, right? We're synthesizing information and we make certain connections. And so these AI chatbots, I think, well, AI technology in general, not just chatbots, are getting better and better and will become better at making those connections. And so I think we're gonna be in a position where either chatbots need, we need to figure out as a way as security practitioners to put the AI to use and benefit or supplement our abilities and give us better capabilities to do threat modeling. And I should say security practitioners or developers to do threat modeling. I don't think they can replace us exactly, mostly because first off, we don't look at just text, right? And now I know image analysis is— there's image analysis capabilities there, but there's a certain amount of interpretation that needs to happen. It's not just It's not just, you know, point A, point B connection threat. And although it'd be nice if it was. And so there's some human element I think still necessary. In addition to that, chatbots are probably not yet very good at that human connection, right? We do a lot of influencing without authority in our roles. And right now the chatbot is there as a guide, but not necessarily as a director. We certainly, we wanna be very careful with that, with allowing that to happen. True.
4:11Chris Romeo So, my thoughts. Yeah, Izar, what do you think? What's, what's your take on this idea?
4:16Izar Tarandach So, if, if Tanya was here, I think that at this point she would ask, what's the word of the day? And I would answer, for me, it's actually two. It's trust and augmentation. And the reason why, why I go there Let's go with augmentation first. For a long time, there was this idea of men are going to be replaced by androids, right? Machines that act and think like us, but we're stronger and faster. Then, and it's not a new idea. It's like 1930s, I think it was the first movie. So it's been going on for a long time. But then at some point, the idea of not quite an android, but a cyborg came in. And that was about augmentation, taking man and giving them tools and extensions to make man better. And I think that we sort of got to this point in the AI journey. And I am by no means an AI specialist, took some classes back in school X tens of years ago. And I think that for a long time we were looking at those expert systems and rule-based systems and all that stuff and thinking, well, this is going to replace a lot of knowledge-based stuff out there. But then we got to this point where right now we have tools that can augment what we already do. And that's what's— that's the way that I try to look at this, this ChatGPT thing, or sorry, large model AI stuff. Because I think that they— I don't see them supplementing, putting us out of a job simply because they can't right now imagine. Perhaps they will be able at some point. But right now, the thing that makes us different from the tools that— the great tools that are out there for threat modeling is that we as people can look at a scenario and imagine what could go wrong. We are not looking at hard attributes and thinking this has happened before in this configuration, therefore it will happen again. We, we can take that extra step and Now to connect to the second word of the day that I will be coming back again and again and again, trust, right? We know that these AIs hallucinate. They themselves tell people, hey, we hallucinate. We go on trips sometimes and we don't know where the hell we're going. So now we paint in a scenario where we are getting a developer that by definition we are saying this is not a security expert. They are not a threat modeling expert. But they are going to ask questions of an AI and run with the answer. And I think that it's a bit of a repetition of the problem that we have with other tools of false positives and false negatives and stuff like that, where we are asking people to use this tool, get an answer, believe it, and run with it, invest time and effort into making what the tool said happen. And can we trust in these tools at this time? More? Yeah. I think that what I'm trying to say is I can see myself, for example, as Adam put on the article and Chris showed afterwards, I can see myself asking ChatGPT something that I would that would take myself in a, in a Google journey of Googling for things and finding resources and figuring them out and putting things together in a nice way. Like give me a list of the threats that the WAF can have and blah, blah, blah, blah, blah. But I wouldn't expect it to create stuff that I'm going to read and be amazed, oh my God, I didn't think of this. Perhaps I didn't think by omission or because I don't know something. But I will still keep that output in such a way that I have to apply my own filters and say how much I trust the thing that's coming out of it.
8:42Chris Romeo But let's put this in perspective though, right? So, you're somebody who has decades of threat modeling experience. You have a mental matrix of threats and mitigations that you literally see threats jump off the page at you because you've done this so many times. Let's turn the course and say, Imagine we're taking somebody who's brand new. They've never done threat modeling before, so they don't have— and I think I can fairly say all three of us have this style of matrix in our head. Like, we can look at a diagram. We don't have to sit and we can go boom, boom, boom, boom, boom. Did you think about this? Asking questions. A new developer doesn't have that knowledge base in their head. They can gain that knowledge base over time. And that— so I see this idea of ChatGPT threat modeling more as an enabling process for people who are trying to gain more knowledge and don't have, they can't afford to have us sit next to them their whole life and just tell them what the threats are and teach them in real time. And so I kind of see it as a stepping stone to a more mature threat modeling approach.
9:53Matt Coles So one thing that I think that you just said, Chris, actually brings up a good question. The type of information that these AI systems are, interacting, they interact with the user because the user gives them information, right? They, they may describe a system or say, I have this type of thing. And if there's a body of knowledge in their, in their model, they can pull that and synthesize it and, and spit it back out. But one thing that we do as threat modelers or developers who are doing threat modeling following, you know, the various practices and guidelines and whatnot that, that are out there is ask questions, right? They, you don't necessarily know what information you need or have until those questions are asked. And I don't know today, maybe one of you know this, do these bots have the ability to look ahead to say, oh, you told me something but not enough. I need more in order to know how to respond to your question.
11:00Chris Romeo Yeah, let me show you an example here, and this will— for those people that are watching this on YouTube, they'll get a chance to see the example. We'll have to describe it for maybe those people who are listening to this. But I did a little, just a little test before we started with ChatGPT, specifically with API Gateway spoofing. And so just for those that are listening that can't see this, I typed into ChatGPT, "List all spoofing threats for an API Gateway." in a table format with columns, threats description and mitigations. And then it spit out a collection of individual threats for me to consider. And so I guess my premise or my hypothesis is this is a database or a dataset that a developer who doesn't have experience with threat modeling, instead of tapping Matt on the shoulder and saying, "Hey Matt, can you tell me all of the threats that I would have to think about for an API gateway?" in regards to spoofing, this is at least a place to start so that the developer could read this list and could say, "Okay, HTTP header spoofing. Attackers manipulate headers. Mitigations are validate and sanitize all input headers or use a WAF or implement API key authentication." "Oh, well, we have API key authentication." But maybe we don't have— maybe we're not validating and sanitizing headers. So maybe that becomes a threat that's more valid for them. So what do you guys think when you see this type of output coming? Is this something that somebody could use to learn from?
12:29Matt Coles I think the answer has to be yes if they're brand new, because we talk about threat libraries, right? I mean, ultimately what we're talking about here is construction of a threat library. And today, you can do that by going to, you know, OWASP, or you go to MITRE, or you go to various other sources, or you can ask, ask the AI to basically do the same thing. And that's really what we're talking about. So the next step, of course, is then to say, well, do I have something that fits, right? I guess it'd be interesting if you, if this was a live example, to be able to ask it, what are the conditions that would result in IP spoofing being a threat that would apply?
13:24Chris Romeo It's thinking right now.
13:27Izar Tarandach While it's thinking, I agree completely with Matt, but again, I'll take it to trust and augmentation. Matt just described a process of augmentation. You can go here, you can go there, you can go to the other place and find your answers. And all that process just happened here with one directed question. What I would like to see on the side of trust is which sources did you use and how confident are you in your answer? So let the AI tell me itself.
14:01Chris Romeo Yeah.
14:02Izar Tarandach Hey, how well do it? Did you do a good job?
14:06Chris Romeo I mean, that's one of the biggest complaints right now though, is that people, we don't have a good sourcing system to know where this thing got its data from. Do we trust that source? There's no provenance of source in, and that's these responses.
14:23Matt Coles And that's the need for the human element then to do the verify.
14:27Izar Tarandach But then we just close the loop with the inexperienced, uh, developer. But first, it's garbage in, garbage out, right? We don't know what the sources are, and that's where the stuff is going to come from. But second, I think that anybody looking into the work that Gary McGraw is doing on security in AI would see the things behind the things that could be influencing the answers that we are getting. So manipulation of datasets, all that, that, that good stuff. Think about this. It's like, what's the old saying that, uh, one of the greatest things that the devil ever did was to convince us that he doesn't exist or something like that?
15:14Chris Romeo Yep.
15:15Izar Tarandach So this is the same thing here. What if, and I welcome our AI overlords, but what if, what if they decide not to tell us about that threat and they decide not to tell anybody about that threat. All of a sudden you're going to have a lot of systems with that threat.
15:33Chris Romeo Yeah. And it's, it just makes me think of, I'm reading a book, you guys may have read it, it's from a few years ago. This is How They Tell Me the World Ends, which is an excellent investigative journalism path through basically security, government, the influence government's had on security. And I just passed through the chapter on Stuxnet and how they, you know, a lot more details than I even ever knew before. But once again, there was examples there where, like, if the government— and this is AI potentially not telling us something— but there was examples there where the government was holding back on certain exploits for their own catalog and letting the easy, the low-hanging fruit be sent to the vendors so they could fix it. And so, yeah, I mean, it all comes down to where the model, where the data's coming from, right? Like, that's— you are putting some trust into the, individual model in this case. And so I'm not envisioning that you would, a developer would use this type of query to create a threat model, stamp it done, it's done, it's out the door, right? There's still, somebody still has to look at it who has some experience with threat modeling just to see. My hypothesis or premise is that this could at least get them some of the way down the road. When you face, and Adam talks about it in this article about the blank, when he first created the threat modeling tool at Microsoft, people had the blank page syndrome. That's what he refers to it as, in that they load the tool, they're looking at it and they're like, "Hmm, don't know what I'm supposed to do now," right? And that's the part I think this, I don't think this replaces us to your point. Like until it can think up new threats like we can looking at the page, then we can start talking about this being a replacement. I'm seeing it as something that will help us to get better.
17:18Matt Coles Yeah. Augmenting our capabilities is—
17:20Chris Romeo Augmentation. Word of the day. Augmentation.
17:23Izar Tarandach Look, look, look at this interesting part of the thing. We, we are focusing a lot on the answers that we are getting from it, right? But, uh, yesterday the three of us were in a call and, uh, we wanted to know the difference between A and B. And I very quickly turned to ChatGPT and said, hey, what's the difference between A and B? And then I put it on the chat.
17:45Chris Romeo Yeah.
17:45Izar Tarandach And then Jonathan Marcel jumped in and said, did you notice that the way that you asked the question added bias to the answer? And I was like, wait, I did not think of that one. So like, is there a difference? Am I telling the AI that there is already a difference? So the shaping of the question is also a huge part of this thing. Now, if it's a dry question like, yeah, tell me, list all the threats done in API Gateway, blah, blah, blah, blah, blah, that's pretty cut and dried. Even though Jonathan, I guess, would already say, who said that this thing has threats? You're already telling the AI that there are threats.
18:27Chris Romeo So it's going to— Well, we can assume. We got to make certain assumptions about life. Right? Like, you can't— we can't live in a vacuum. Well, but we're—
18:36Izar Tarandach Can we?
18:36Chris Romeo Yeah. I mean, we're not— but we're not leading— I wouldn't call that bias. That's just a binary answer. It's one. We know there are threats in everything that's been built. We can— so we can assume that as a starting point.
18:49Izar Tarandach But again, then comes the question, can we jump one level up? Can we stop and look at that imagination thing? So just this morning, I was playing with something completely unrelated. And I asked ChatGPT, hey, in a business process of somebody trying to buy something from a web store, what are the possible business threats that you're going to find there, business-level threats? And it came out with an answer. And the answer was well-structured, it was well put, and it was at the development level. You have to have HTTPS because you can have somebody checking what you're buying. You, you have to have all the usual suspects, right? So it went back to what it read somewhere else and brought it happily back to me. It did not think about, hey, what else?
19:43Chris Romeo Yeah.
19:44Izar Tarandach So my question now is, fine, we are introducing, uh, uh, uh, AI into the arena. Should we now talk about CIA plus privacy? Let's trust.
19:59Matt Coles We should.
19:59Izar Tarandach Trust now a new security value?
20:01Matt Coles Yeah, we should. And actually, I wanted to raise this topic, right? That what are you giving up or what are you losing control of by asking questions of the chatbot?
20:16Izar Tarandach Nice. Yeah.
20:16Matt Coles So for example, we know that it learns from input or it, well, I should say the original ChatGPT-3 did not, to my understanding, right? It has a large language model, but it didn't, it wasn't it wasn't really adaptive in that regard. But the newer versions, GPT-4 and some of the other systems, almost certainly learn from behavior. They learn from the questions. They can be potentially provided new information, right? Either real-time or, or, or sort of at question time information. More importantly though, we don't know in all cases what they're doing with your, with your data. So if you have a developer, at their desk in a company and they ask about their products or application they're working on and they describe it to the chatbot, does that become now part of the large language model and have you lost control of your IP?
21:13Izar Tarandach So Microsoft in their Copilot thing, just jumping forward a bit, they are very explicit on two things. What they call responsible AI. So your data is never going outside the boundaries. It's never going to be added to everybody else's. So the world that you're going to be looking at is still very, very limited to you. And they mention the fact that, yes, we will get things wrong at some point. So they're already adding the but for the trust.
21:42Chris Romeo Let's transfer to that article here. Maybe you can set that up a little bit because you mentioned Microsoft. Security Copilot, but not everybody may have seen or heard kind of what that thing is. So if you can kind of set the stage for us and then you can lead us into any of the points you want to discuss.
21:59Izar Tarandach Right. So this one came out. It's not actually an article. It's more of a launch. And it came out this week. And it's basically Microsoft doing apparently very well The product is not available, but there is a video in there doing what we have been doing for the last 6 or so months, throwing things at ChatGPT and seeing what comes out. Okay. So they are proposing to do 3 things to simplify the complex, meaning during an attack, if you have complexity, it's time, it's expenses that you don't want to have. So they are bringing data from a lot of sensors that they have and synthesizing the whole thing. And they're proposing to catch what others miss, meaning that they are going to do the triage of the signals for you. But then comes the third one. They want to address the talent gap. Sounds familiar to anyone? And they say the demand for skilled defenders vastly exceeds the supply. Help your team make the most impact and build their skills with step-by-step instructions for mitigating risks. Well, now that, that takes me back to the trust. This thing now is going to take people by the hand and mitigate risks.
23:20Chris Romeo Yeah. And I, I'm reacting to, uh, you might've seen the look on my face of, yep. You know, the talent, they're gonna, they're gonna close the talent gap for us. Come on, man. So that's somebody, somebody in marketing got a little too carried away on this website.
23:33Izar Tarandach That's not even the part that, that worries me. The part The part is that this thing is going to give step-by-step instructions for mitigating risks. So first of all, it's going to identify the risk. It's going to report that risk to people who will say, hey, I don't have a lot of experience, but I don't need because this thing is going to take me by the hand. Then it's going to tell you step-by-step what you have to do. So from its point of view, it closes the risk, but we know that the trust may be iffy here and there. So, did it really close the risk, or is it just not identifying all the risks related to that specific thing?
24:13Matt Coles Well, I think— so, I don't know all the details. I just sort of read an article on this. What I understand is, first off, when we're talking about defenders and threats and risk here, we're talking about SOC analyst functions. Detection, right? We're talking about Security Operations Center, looking at sensors, looking at events. We've used SAM and single pane of glass in our, in our swear jar before. And, and, but that's, it seems like that's what, what this Copilot is here for, at least in the current iteration, is, is around automating sensor detection, incident response, and handling. And so closing the risk is procedure XYZ. Go pull your network cable, save all logs, start forensics, right? And implementing procedures, you know, rote step-by-step instructions should be, actually should be something that our tools already should have had capability for, right? It would be nice if a SIEM, I don't know if they do this today, but it would be nice would a SIEM system be able to, hey, I got an alert, follow this procedure, right? And insert, you know, this was detected at this time and all that. And, and basically create a step-by-step guide. Why did we have to wait till now to get this? But if that's what it's supporting, if that's what it's going to bring to the SOC analyst, then they, the SOC analyst can be busy doing other things most of the time.
25:46Izar Tarandach But that's my point. That's something that for all intents and purposes we could do without AI. We don't have to involve AI. My point is exactly that, that AI is bringing in that aspect of trust, and I'm not sure we are there yet. To give an analogy, it's the same thing as like, we don't need pilots anymore. We can put anybody in the cockpit and an AI is going to write a bunch of checklists and they're going to follow the checklist. Only that in the checklist for landing, it forgot to put the item drop landing gear. So yeah, I'm gonna follow the checklist and this plane is going to end up going down the runway, but oops, no landing gear. Why? Because of the input, because of I don't know what.
26:26Matt Coles But I think that, that— I mean, that goes down to your augment versus and trust discussion you had earlier, right? A company would be foolish to take the leap from I need very skilled security operations center people, analysts, and incident responders and handlers, and I can throw them all away because, well, the AI is going to take care of that. That's a quantum leap, right? It's not the right term.
26:54Izar Tarandach It was fine because companies never did anything foolish.
26:58Matt Coles But baby steps here, right? If we think of this as new additional value-add functionality onto systems that are already in place, then you can take that same highly skilled analyst and make— allow them to do other things like start doing preventative work. Or maybe you don't have a lot of your highly skilled SOC analysts, and now you can take somebody with the ability to follow procedures, the ability to understand what to do if something goes wrong outside of that procedure, but not necessarily be fully at the analyst level. And give them that leg up into that role.
27:37Izar Tarandach But now comes the question: Does that give me somebody who is highly skilled at following checklists, or does that eventually give me a highly skilled analyst?
27:46Matt Coles That's a great question.
27:48Chris Romeo Yeah. And I don't— I've read, honestly, I'm a 5-minute expert in Microsoft Security Copilot. Everything that I could read in the last 5 minutes. But, To me, based on the description, it doesn't— it looks more like it's just a data-crunching platform that's going to help me to bring my Intune data, their kind of threat intelligence, and then I don't remember what the third one was, some other leg. What was it?
28:15Matt Coles It's kind of like predictive analytics. It's kind of like predictive analytics, which we've had for a long time now on, you know, machine failures and other things.
28:25Izar Tarandach Yeah. But it's putting a really nice face to it. Ooh. All of a sudden you have—
28:30Chris Romeo Ooh, a single pane of glass.
28:32Izar Tarandach No, now you have a single pane of partner. You have a partner to work with. You can ask questions. You can have answers.
28:40Matt Coles Is this Clippy all over again?
28:42Chris Romeo Oh, I hope so. I love Clippy.
28:44Izar Tarandach It's what?
28:45Chris Romeo Clippy.
28:46Izar Tarandach Clippy.
28:47Chris Romeo Yeah. We want Clippy to come back.
28:49Izar Tarandach He's trying to defend from ransomware. Can I help?
28:53Matt Coles So I have to ask you a question. Because you mentioned, you know, that— and I don't know much, again, I'm just learning about Copilot, but you mentioned that it, you know, your data stays within your data, within your zone, but they're using their trillions of sensors to be able to understand what's going on. Which is it? Is it assuming that you're in an Azure environment? It doesn't work outside of that environment?
29:20Izar Tarandach I'm just reading what the NICE website is telling me, and it says, "Train Security Copilot with confidence. Your data is protected by the most comprehensive compliance and security controls," sorry, "in the industry, and it's never used to train other AI models." That's fair, but whose data is used to do threat identification?
29:46Chris Romeo Yeah, I mean, this thing had to be trained somehow.
29:48Izar Tarandach I would say that Microsoft, I would guess that Microsoft data, they sort of like have a lot of it.
29:53Chris Romeo Yes. I mean, the, the, the, as we've said before, the answer you get is only as good as the training the model received.
30:01Izar Tarandach Mm-hmm.
30:01Chris Romeo And so if they, now Microsoft of any company on earth has close to the most amount of data for these type of situations that they could use to train models and things, but Yeah, I don't, I mean, I think this is definitely one to watch. I think it's, yeah, that's, I'm curious to find out if it's how much AI and how much old school security technology that's being painted using an AI view to get people's attention, you know, or is this really AI for the SOC?
30:36Izar Tarandach Security dollars are expensive, you know?
30:39Chris Romeo That's true. That's true. Well, let's— I mean, we've talked about threat modeling and ChatGPT. We've talked about Microsoft Copilot. Let's talk about shutting this whole thing down. Not the podcast. Not the podcast. Shutting this whole AI thing down for 6 months. So, Matt, give us kind of a high-level view of this letter that you pointed us to, please.
31:00Matt Coles Yeah. So, Future of Life Organization, which I'm really not familiar with. So, I've seen this. Actually, somebody else forwarded this to me, and I was curious, so I dug took a dive into it. It's basically an open letter to AI researchers to request that there's a pause in development of these highly accelerated AI and expert systems for various reasons. So currently, as of today, there are 1,922 signatures added to this open letter, including some pretty famous names like Wozniak and Musk, and quite a number of folks. There is a long list in the actual article letter here. And what they are asking or suggesting is that the advancement of AI has exceeded our ability to manage it. Or if not now, in the very immediate future, will have lost control. And that therefore we should pause development to really understand what the impact of AI is. And I think the areas that people are, the sensationalist areas of this, right, is, oh, is ChatGPT and other AI systems going to take my take our jobs? Uh, are you going to put millions of people out of work? Um, are you going to have them now be in charge of critical infrastructure and/or weapon systems? Uh, and, and we should all look at that and go, well, they're already today involved in many of those things, and we're generally okay with that. Um, but I think this is where the— this group is— wants to make sure that the human element is included in the AI discussion in a way that they believe it's not being done today. And obviously we're talking about this. I have a little, just have read the open letter, right? I'm not involved with it. I don't know much about this group. I don't understand what their, any political leanings or technical leanings or otherwise are. So I'm sure we'll get comments on this. Hopefully good constructive comments, would love to learn more about this. But based on the open letter, let's pause development to make sure that the I's are dotted and T's are crossed with respect to the ability of us, of humans to be in control of the technology, not the technology in control of humans.
33:47Chris Romeo I saw a joke in reference to this letter that was basically saying, this is, and I'm not going to even name who the companies were, this is a few companies trying to catch up. This is their, their way of trying to catch up on the market is like, let's have an open letter and shut this thing down while we develop in the background. And then in 6 months we have a product ready to compete with, with what OpenAI is doing. So, you know, with, with, as they say, every little bit of humor has a shred of truth. Yeah. That's what makes it funny.
34:16Matt Coles And so, yeah, I, I wonder, like, you know, I, I questioned the 6, the 6 months seems like an arbitrary number. Does it, Is that really going to stop somebody from creating a system that is destructive? But I also don't expect us to suddenly be out of work in 6 months, right?
34:36Izar Tarandach Look, my first opinion on this, that's adorable, right? You look at the list of signatories, okay? And you have Mr. Musk, who, if I remember correctly, was one of the early investors in OpenAI.
34:58Chris Romeo Yep.
34:59Izar Tarandach You have yum, yum, yum, yum, yum, yum, a bunch of conjecture, Pinterest, Ripple, Getty Images. Getty Images has a problem with generative images. I could not believe it when I read it. And then you have the inventor of associative neural networks, who is, I believe, one of the technologies that's being replaced. But at the end of the day, you have like a mix match of people here that's, in my opinion, average to the good old CYA, right? They are covering their asses. They want to be able to go back and say, but we warned everybody, because they are asking for 6 months For what is it?
35:50Chris Romeo Uh, not training anything above the GPT-4.
35:56Izar Tarandach AI developers must work with policymakers to dramatically accelerate development of governance systems.
36:03Chris Romeo Oh boy, governance, governance, government oversight.
36:06Izar Tarandach How many things have worked in the past along this line? 6 months. In 6 months, perhaps they're going to decide what's the name of the committee. And where the coffee is coming from.
36:18Chris Romeo Right?
36:19Izar Tarandach So 6 months at the speed of development today means absolutely nothing. And to contrast and compare with Chris's joke, I can imagine a near-peer nation right now, their military labs, telling everybody, okay guys, go home. Somebody wrote a letter and we have to stop for 6 months. Not going to happen. These things are already weaponized. People already seen the next chapter coming and it's an arms race. You don't pause an arms race.
36:57Chris Romeo Yeah. Let me quote from this letter as well. This does not mean a pause on AI development in general, merely a stepping back from the dangerous race to ever larger unpredictable black box models with emergent capabilities. What the heck is an emergent capability? Are they, I mean, is an emergent capability what I think it is?
37:20Izar Tarandach It's, I think it's you learning that your AI can do things that you didn't expressly put in there.
37:27Chris Romeo Yeah. Okay.
37:28Matt Coles You may give it access to the internet and then what is it? What capabilities does it gain?
37:34Chris Romeo So it's basically the, is it Superman 2 or Superman 3? Is the plot that they're laying out here where Richard Pryor, uh, War Games didn't— no, I think it was, it was one of the Superman movies where Richard Pryor was using the computers. Basically, I think got control of, of it, got out and got, got access to this data.
37:56Matt Coles All of the moves you just mentioned are the same thing, right?
37:59Chris Romeo Yeah.
37:59Matt Coles Except for maybe not War Games. We actually don't know. I guess, do we know that War Games would have actually launched Oh, that's true.
38:06Chris Romeo I forgot there was a game.
38:07Izar Tarandach It's the movie that launched my career.
38:10Chris Romeo No, that's good.
38:12Izar Tarandach I didn't even—
38:13Chris Romeo I thought about that. Would you want to play a game? I mean, come on. That was the same kind of basic thing. And there it is highlighted in Isar's office right above him by the Threat Modeling Manifesto and a great picture of his book that he and Matt wrote.
38:28Izar Tarandach My wall of glorious things.
38:30Chris Romeo Yeah, so I mean, I guess it sounds like we're all kind of coming to the same conclusion. It sounds like none of us are signing it. Is anybody going to sign this out of our group here?
38:38Izar Tarandach I can sign because doesn't mean anything.
38:41Matt Coles So that's true.
38:42Chris Romeo I'll sign it. I mean, if they'll put my name up next to Elon's, I might sign it.
38:45Izar Tarandach Well, no, no, no, no, no, wait. I can't sign because then the AIs are going to see that I signed and they are going to make my ration smaller.
38:52Chris Romeo That's right.
38:53Matt Coles You're—
38:54Chris Romeo when we get the Silent Green. When we're in the Matrix and we're in those pods, we won't even know anyway. We'll We'll be attached to the machines.
39:01Matt Coles I just want steaks. Yeah. Now, having said this, I do want to raise something along the same lines. You know, there have been some conversations over the past couple of years about the rise of intelligence systems. And I want to be careful here. The rise of intelligence systems, I think, is what started the conversation, but this has been happening for for decades around automation and humans, right? And ultimately AI systems are that, yes, they are artificial intelligence or machine learning or whatever term we want to use today or generative model, large language model-based generative texts, predictive text. GPT is generative predictive text. It is not a thinking system specifically. But ultimately, they're all methods of automation of previously mostly manual tasks, right? We do industrial automation today with robotics, right? We have cars that are able to stay within, within reasonably well-defined boundaries so that they don't get into accidents. We have autopilots in planes. Humans adapt, and we, we need to, we need to think about We need to make sure that we are doing— personally, I think we do need to make sure we are keeping abreast of these technologies so that we make use of them appropriately, right? And do skill retraining where necessary. Understand that there will be a need for some sort of support systems in place to be able to support industries if they disappear. And they're not going to disappear by by any stretch with GPT alone, right? Security Copilot isn't going to eliminate the need for SOC analysts, right? It's going to make SOC analysts more effective, most likely. So we, I think hopefully, and I, we trying to avoid introducing direct politics into this, into this podcast, because boy, I don't want to step on that landmine. But, you know, The open letter is a good, is a good position paper, but is it really going to be effective for what needs to be done? And the answer is probably no.
41:27Izar Tarandach No, cuz it's performative. Nobody's going to even think about following what's, what's written in here. And even if they do, who says that everybody does at the same time? And somebody got—
41:37Chris Romeo they're suggesting government step in if people don't follow. You know, I don't know what, you know, and let me, let me read one more sentence that kind of caught my attention here cuz I've been, as I've been, we've been talking, I've been reading and thinking thinking about this more. And, it's right towards the end, it's like the second to last sentence in the letter. It says, "Society has hit pause on other technologies with potentially catastrophic effects on society." And then, it has a footnote down to number 5. The footnote 5 says, "Examples include human cloning, human germline modification, gain-of-function research, and eugenics." And so, human cloning is kind of the one that I think is the most easy to understand. I can't claim that I know what all those other things are on that list, but human cloning, I understand what that is. And there has been a moratorium on human cloning, but I guess is that the same thing as building, you know, continuing the AI development?
42:34Izar Tarandach I'm sorry. Has there been a moratorium or has there been a moratorium declared?
42:42Chris Romeo That's a good question. I don't think I can answer that.
42:44Izar Tarandach This is transformative technology. Like, governments, they don't work on ethics, they work on interests. And AI is a leg up in this competition that we live in.
42:56Chris Romeo Yeah.
42:57Izar Tarandach Nations are not going to drop interests for the goodness of ethics. Otherwise, we wouldn't be in the situation that we are now with uh, Earth, uh, climate change and, and, and all that stuff.
43:10Matt Coles And, and real realistically, sorry to interrupt this, or realistically, these new technologies. Now I want to be, yeah, the, the list that you have here, right? Um, you know, eugenics is obviously that, that's, that's the, that's probably the worst one here, right? Which is, yeah. Uh, maintaining a particular, maintaining human society in a particular way. Based on, you know, addressing issues. Well, sorry, not addressing issues, addressing or influencing who and who and— wow, it's so hard to talk about. I don't want to bring it up because it is really bad that we should not be talking about this. But I mean, we have to, right? That Realistically, that is, that is bad and does need to have a moratorium because of what that does. It's not supportive, right? It's, it's a biased technology, not supportive of, of making humans better. All of the other systems, not this list, but all of the other systems, I think that, that are happening around robotics and driverless cars or And all the stuff that preceded that, right, around safe automatic safety systems and real-time monitoring of a ton of datasets, you know, to be able to do real-time analysis, all of these things are augmenting, to Isor's point, human capability. And as long as AI continues on that path, that's a good thing. And should be looked at as a good thing, in my opinion.
44:55Chris Romeo Yeah, I think that's a good, uh, that's a good place to end this episode. That was a good— that was a nice summary down to a, you know, kind of a conclusion as far as how, uh, how we can walk away from this. So, uh, this was our first, uh, Convergent AI episode. Probably won't be the last one. We'll probably— I think this is going to become more and more ingrained in the conversations that we're having about how security gets done. And I think we're going to have to keep revisiting this stuff.
45:25Izar Tarandach But it's probably the last one for the next 6 months, right?
45:28Chris Romeo Yeah, because there's going to be— it's going to be— we can't even talk about it. We don't want to get in trouble.
45:34Matt Coles We're going to get so much hate mail on this.
45:35Chris Romeo I hope so. I hope so.
45:38Izar Tarandach From ChatGPT? Yeah.
45:41Chris Romeo It'll be just random. It'll be from like made-up names, like, you know, Bill Smith, Jane Smith. Wait a minute, these names don't sound real. Come on, ChatGPT, use a better generator. Yeah, we'll see. But hey, so thanks, folks, for listening to this episode of Security Table. We will talk to you again soon.
Transcript supplied by AssemblyAI. It may contain transcription errors.
Originally published as The Security Table. Part of the AI Security Table archive.
