44 minSeason 1, episode 23

Security Champions as the Answer to Engineering Hating Security

With Chris Romeo, Izar Tarandach, and Matt Coles

Teams, careers & security culture

What happens when engineers transform into security champions? Is this beneficial, and what are the implications of this transformation?

Episode title image: Security Champions as the Answer to Engineering Hating Security

Listen to the conversation

About this episode

What happens when engineers transform into security champions? Is this beneficial, and what are the implications of this transformation? Izar reveals his transition from a naysayer to a supporter of security champions, and Chris and Matt seek to understand his current position. They explore the position of Security Champion and discuss the components of a good security champion program.

Matt defines security champions as developers with influence who can be a bridge between security and engineering. They receive advanced training and bring resources to their team to lead them to effective threat modeling. While security champion programs may have potential pitfalls, such as overloading team members, good security champion programs should benefit the individual and the business. Chris emphasizes the importance of providing opportunities for growth, learning, and networking to make the program appealing to potential champions.

With the potential issue of champions leaving an organization, they highlight the need for companies to keep up with salary expectations as champions grow in their roles. They also touch on the challenge of preventing security champions from being disliked by their team once they transition from being developers.

There are several resources for those interested in building a Champions program, including Dustin Lehr's Security Champion Success Guide and Chris Romeo's Security Champion Framework available on GitHub.

The episode concludes with a call for listener feedback and input, emphasizing the hosts' desire for an interactive and engaging conversation with their audience.

Follow AI Security Table:

➜ Home: https://securitytable.ai/

➜ X: https://x.com/AISecurityTable

➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/

➜ YouTube: https://www.youtube.com/@AISecurityTable

Transcript · 44 min conversation

0:07Chris Romeo Hey folks, welcome to another episode of The Security Table. This is Chris Romeo, joined by Izar Tarandash and Matt Coles. And we're going to talk about, well, we're kind of going to make a connection to something that we talked about last time, but in our pre-podcast discussion, Izar shared it in such a crisp way, I was like, I'm gonna hand the reins over to him and let him tee up this particular conversation for you.

0:38Izar Tarandach So, in a previous, I think one before the last episode, we spoke about do engineers hate security people? But what happens when we turn engineers into security people through that magic formula that— Oh, great.

1:04Matt Coles We're going to get demonetized. You know that, right?

1:08Izar Tarandach No, it's like less than 30 seconds. But yeah, but what happens if we turn engineers into security champions? Is it a worthy endeavor? Does it give what we think that it does? And disclaimer, I am known as the naysayer of the security champions, but, uh, through the good offices of people like our Chris and Dustin Laird, and, uh, I sort of changed my mind a bit. So let's see if we can change my mind a bit more.

1:38Chris Romeo That was a— that's a big move too. I mean, it reminds me of a very quick story that we've already told once. I'm going to tell it again. When Izar and I were at Lascon And Jim Manico gets up and he says, "Hey, I think threat modeling's actually a good idea." And the camera pans to Izar and I, we have fallen out of our chairs. We are laying on the floor. Izar looks at me and he goes, "Wait." He literally did this. We didn't fall out of our chairs, we were standing. He literally looked at me and he goes, "Wait, what did he just say? Did he just say what I think he said?" Did he say what I thought he said? I'm like, "I think he just said threat modeling's a good thing to do." And Jim had been, very much a naysayer of threat modeling for a number of years. And so that was a powerful move. So let's dive into security champions, though. And Matt, why don't you tee up for us this idea of security champions? Let's just assume that people listening, maybe they don't know what we're talking about when we say security champions. So give us a working definition, if you would.

2:31Matt Coles Right. So security champions are folks who we tap as security teams Security champions are those that we tap for additional responsibilities and specifically to be a bridge between a security team and an engineering team. So these are folks who come, who should come from an engineering team, who either have security experience or have desire to learn about security and act as a liaison and a boots on the ground, so to speak, between, with the security team sort of providing direction, and the security champions taking action in their local environment with their development group and with their product managers and others to implement the security program, whatever that is. So I think that in a nutshell is what we generally consider as for security champions. And, you know, there's a whole gamut of the people who who end up becoming security champions, right? They get— ideally, they come from engineering teams, right? They're technical folks who have not only technical capability, but also influence, you know, influence among their engineering orgs so that they can, for instance, educate people on the value of threat modeling and execute on that as an activity without necessarily having to pull in the security team, although you could. They can, and as a liaison, they should be able to bring in resources. But we generally train them. They get advanced training. They get, you know, first crack at tools or, you know, new techniques or whatnot so that they can take action within their environment doing all the local coordination. That's how— I think that's generally what we consider across the board here security champions.

4:28Chris Romeo Yeah, I think that's a good definition of security champions. And I'll expound a little bit more on the why do we need security champions? Because I totally agree, Matt, with what you're saying about kind of what they are, what their job description or kind of focus is. I think it's important for folks to understand why do we need security champions? Because, and the fact that I always go back to, there's one thing that I look at in the BSIMM report. For a year it comes out. It's only one thing. The rest of it, I just throw off to the side. But I'm always curious to see what's the ratio between the number of developers to the number of what they call software security group members. Now, I hate that term software security group. Like, I don't want to be an SSG, but it's always an interesting metric because then I always have to remember, but wait, these are the companies that take security and AppSec and software security so seriously that they'll spend a lot of money to be able to do, to be assessed by a commercial assessor who's coming in and going through a formal review and then generating a report and a score and everything for them. And so, BSIMM 12, it was 135 to 1 developers to software security group members, but that's for the people that really take it seriously. So, it makes me wonder, is it 200 to 1? Is it 500 to 1 in different companies? It probably is in some of the bigger companies that exist out there. And so, for me, Champions exist as a bridge between my security team and my engineering team because I'm never going to be able to resource— I don't care what anybody tells me, and there are people that argue with me about this, to say, yes, you can resource a team big enough to put security team members and distribute them in engineering. I've never seen a company where there was enough room in the budget for me to hire 100 people just to be my connecting connective tissue into engineering for AppSec. And that's— I mean, 100 people, it's just not going to happen.

6:26Matt Coles So, well, and in fact, Chris, just to add to that, in our space, well, I'll say mine and is ours, right? Well, previously, I guess, is our view with product security. There aren't that many. There probably aren't that many people in a particular geographical location. Now we do remote. Even still, they're just aren't enough of us, right? We have a cybersecurity skills, you know, a workforce shortage in general. Finding 100 people for a security team outside of just that role is a challenge.

6:57Chris Romeo Yeah.

6:57Matt Coles So, uh, yeah, there's a scalability problem there.

7:01Chris Romeo So I'd love to hear, Isar, your journey from being a naysayer to a slightly less starting to think this is a thing that could maybe work. Like, because I want to, I want to unpack, I want I want to unpack the challenges that you had with security champions. I think that's a great way to explore this in more depth because for a long time you were the one saying this doesn't work because of X, Y, and Z. Yep.

7:27Izar Tarandach So, much like Matt's definition, I find that whenever we start talking about security champions, Somebody very quick jumps up and says, uses the expression force multiplier. So there aren't enough security people out there. Software is eating the world. We need more people into more software. Hey, let's get people who are already in the software, turn them into sort of security people, give them training, give them belts. And these people are going to act as force multipliers of the security people who are embedded into these teams. So first of all, my initial reaction is, okay, so we are asking people who hate us to turn into us.

8:28Matt Coles Oh, it gets worse than that. Let's talk about that in a bit, but go ahead.

8:31Izar Tarandach I need to give them damn good incentive to do that. The incentives that I have seen out there, frankly, I have not found a single one that, if I were an engineer, would cause me to do that. Couple—

8:51Matt Coles so can I actually— can I, can I just jump in here? Because I— before I forget the thought, it isn't that— there's— do you, do you find in your experience, in your journey here, that it isn't just that you're asking engineers who probably hate the security team initially to become security because their role changes if they become a security champion, but often they're voluntold to do so?

9:21Izar Tarandach Yes, that's another point. That goes when the incentive is not good enough and somebody jumps. Very quick, somebody is told, now it's you. And many times that happens. I don't have numbers, but from observation only, anecdota, as they say, it happens to the least influential, newest member of the team. Exactly the person who does not have the voice to push the initiatives that we are asking for.

9:53Chris Romeo Yeah, right. And I would say—

9:54Izar Tarandach Wait, wait, wait, wait, wait, wait, wait. There's more.

9:57Matt Coles There's more.

9:58Izar Tarandach On top of that, when we do find somebody who is influential enough, that person usually comes already with a big workload. And now we are putting them in a tug of war where we say, I want more of your time for security. You still have to do all the stuff that you were doing before. I'm not giving you any kind of incentive, reward, or otherwise. And by the way, you are going to be accountable for this. I've seen people burn out and leave for less.

10:27Chris Romeo Yeah. And let me give you, as someone who's been a big security champion proponent over the years, I ran Cisco's security advocate program for about 5 or 6 years, turned it into a global conference series where we had internal AppSec product security-focused conferences happening in the US and San Jose with 1,000 people there and in Bangalore, or India, Shanghai, and London, like all these things. As somebody that's kind of been down this road and seen what it took and tried some things that didn't work and tried some things that did work, the biggest takeaway that I came up with is you have to make the champions program about the champions. You have to think about it, what's in it for them? So many champions programs that exist out there try to be force multipliers, but they think about it in the terms of the business. Okay, Matt, you're going to start a Security Champions program. And so, you tell me as your leader, what's in it for me as the business? What am I getting out of this? And they build their programs according to that structure. And what happens is you end up with this program that, to your point, there is nothing in it for the developer, the engineer, other than, ah, my arm got twisted, I got to go be a champion because my boss told me I have to be, and this sucks. Whatnot, because we didn't flip our mindset around and say, okay, let's build a champions program. What is going to be attractive to developers where they're going to go, I want to be in that? And I can tell you this, after doing this for 5 years at Cisco, I took, I took a dying champions program from 20 people to about 500 when I left. And the way I did it is I created something that people wanted to be a part of. They— I went from in the early days walking around the campus and around San Jose and various Cisco buildings, knocking on cube walls saying, is are, will you be a security advocate? It's going to be really cool. And getting a small group of people to do it— right before I left Cisco, people were— you had to go through a series of requirements to get in. You couldn't just call me and join the group because we had matured to the point where people saw it as, I want to be one of those security advocates. I want to be associated with that brand. Those are people that are making stuff happen here inside the company. They're getting enhanced training and opportunities. And I got to watch a number of people that were champions become security professionals. They left the world of engineering to become security professionals.

12:54Izar Tarandach Oh, that's my next point. But first of all, why is it that I hear vibes of, do you want to be in Glee Club?

13:00Chris Romeo But I can't sing. I can't sing at all. So Glee Club is not for me.

13:06Matt Coles But not Fight Club.

13:07Chris Romeo Fight Club I can do.

13:09Izar Tarandach We don't talk about that one. But let's talk for a second about that. They moved to security people. So that's the thing. That's the one thing that I found that for some people at the end of the road, there was some attraction in there. People who would come to me and say, listen, I'm really interested in starting a career in security, but I have no clue where to start. And then my next question would be, would you consider start as a security champion? That makes absolute sense. But consider their journey. At the end of it, they are looking already at an escape point. So by definition, when you get people to the level that they can actually be really, really, really useful— I'm not discarding their, their usefulness before that, but really, really, really useful— when they can really be a force multiplier, they will either join the security team or they will pop out and go be a security person somewhere else.

14:08Chris Romeo This was a percentage, right? I had a percentage, a small percentage of people that went all the way through the program, through a master's degree that we set up at San Jose State that was tailored for Cisco for the security advocates team. I had a handful of people that went through that and became, and transitioned to security. I would say that the way that I approach it for the average developer is today when I'm recruiting a security champion, the way I sell it to them, because it is a marketing and sales move, let's just be honest. It's about sales. I tell them though, you're a developer now, you're a senior software engineer, you're good at writing code. If you layer the application security knowledge and skills that you'll get as a champion together with that, you're in the top 5%, maybe the top 1% of your industry because you're a developer who understands AppSec. You're not an AppSec person who's a developer. And when somebody gets to that level and they can achieve that level of knowledge, I mean, I just tell them, You're becoming it. You're entering unicorn territory right now as a developer that's got that foundation that doesn't want to be a security person, but understands the principles that we all prescribe and talk about all the time and try to get people to adopt and adapt.

15:18Izar Tarandach But, Chris, unicorn here is a good label because we are talking about this mythical creature that a lot of people don't believe exists and most of them are not looking for.

15:33Chris Romeo So, I use the term— What I'm describing though is a throwaway unicorn. I threw that in. Let's not use that term. But think about it. Let me ask you this question. If you have a senior software engineer who has spent a few years learning about AppSec but still just wants to write the best code on earth and build cool products, aren't they at the top of their industry as a software engineer? Isn't a software engineer— So, you don't think a software engineer with AppSec knowledge and skills is better than a software engineer who knows, doesn't care about or know about security?

16:05Izar Tarandach If you measure to be at the top of the, of their industry by how wanted they are by potential employers, we have already agreed in past episodes that employers are not asking people, do you know AppSec? And they don't consider that an added value when bringing engineers in. So it might make them personally a better engineer, because as we agreed in previous episodes, security code, secure code is quality code. So yeah, they're going to be writing better code. Is that going to give them a leg up when they go and interview? Questionable.

16:43Chris Romeo I mean, I think it only gives them, it's not a negative. It's certainly not going to be, it's certainly not going to take you down. And I would say a progressive security-first company, I think we're going to see more of this is going to become a bigger deal over, you know, I mean, we've talked how many times about all these regulations. And all these documents that are coming out that are going to do nothing but drive security into the forefront of everybody's mindset.

17:07Izar Tarandach Agree completely. That's what I'm going for.

17:09Chris Romeo I think this prepares these people to be— and once again, this is my sales pitch to them. This is what I think they're going to get out of it is, why should I be a security champion? Well, we're building a program that'll help you to move and grow and improve your career over a period of time. I think that's a real, a real benefit.

17:28Izar Tarandach It's, it's a bit equivalent to somebody asking, why should I have an MBA? It's something that's years in the future. You're going to be trained for this. It's going to take time. You'll be there at the end.

17:41Matt Coles Yeah.

17:42Izar Tarandach On the way, they're going to have to pay their dues. Is that carrot sweet enough to dangle in front of them? To justify the fact that all those years they are going to have that tug of war between their developer work and now the security work. They're going to have to, let's be honest, go through the whole horror cases that we as security people go and that we spoke about, getting time, getting resources, convincing people, cajoling people, asking people, imploring people. Doing all that good stuff.

18:21Matt Coles And by the way, that's what we expect of them. That's the tasks that we push to them. And hopefully, an organization that has implemented a security champion program is backing that up with resources, with some amount of resources, whether that's direct support, handholding, training, tooling, whatever, right? Ideally, it's not, "I throw it over the wall," Although I imagine that happens.

18:48Chris Romeo But to address, to Izar, to your direct point, I would say to use a term you used earlier, I have anecdotal data to say that yes, it does. It's not, it's not, I don't have industry-wide data. I haven't been able to apply this model to 10 different companies yet and see if I can get the same result. I did it with one that was a, that's a big company that tends to be a challenging case study because it's engineering first, engineering, at least in the days when I was there, at Cisco, engineering ruled the place. Like, you didn't tell engineering what to do. You could suggest. Sometimes they would laugh at you. Sometimes they would say, "That's a good idea." But they had that type of power. And so, but yeah, it's anecdotal at this point. It's not a model that I can say is— I can't prove that that is the case everywhere. But I think over time with what the attention we see with champions programs, I think we're going to have more and more data as people are getting into this and now they're starting to talk about what's happening in Champions programs. I think we're going to see if there's some reality here or if I got lucky the one time I did it.

19:55Matt Coles And I just want to highlight, you know, we've been talking about Champions primarily from an engineering standpoint, from a developer standpoint. I do want to highlight that, and strictly we've been talking about software, but I do want to just highlight for folks who may be interested in learning more about what Security Champions are, either within your own organization or just in industry in general. It also works for hardware. It also works for vulnerability and incident response. Well, vulnerability, I'll say vulnerability response. I'm not sure about incident response, but certainly would work for those areas. So, not just development, software development, but other aspects of system development, hardware, software, and then both pre- and post-release.

20:42Izar Tarandach So where am I in, in that journey right now? Where did you guys get me to over the, the last few years? I think that I have abandoned the idea of having a bunch of mini-me's, and I have adopted the idea of having a bunch of minions. So to use the word that Matt used earlier, liaisons. My thing now is security liaisons. For that force multiplying thing, I want in every engineering team somebody who is a point of contact with the security team, in the sense that I'm reducing the workload for the security team to have one point of contact where they can go with their questions. Where they can go with their requests. What's going to happen internally in the engineering team, it's their thing. I'm transforming that, that team into a black box. I have one single interface. I go to the person and I say, this is what I need to happen. Can you help me make it happen? If it's a question, they'll know way better than me who's best equipped to answer. If it's a request, they know way better than me where the team is and if the team can stand by the request or not. By not having any, uh, preordained expectation of them rising their own security knowledge, security-minded and security abilities, I am not putting an undue load on top of them. But at the same time, I am prodding them and I, and I am giving them enough material to understand why I'm asking the questions that I'm asking, to understand why the things I'm asking for are better than not having them, right? So in separate from them, I find those people that want to be mentored, that want to be taught, that want to, to be challenged, to go and learn more security, because perhaps those are the ones that the carrot of having that thing at the end of the couple of years tunnel is interesting.

22:57Chris Romeo So I guess let me, let me ask a couple of clarifying questions just because I'm curious. I haven't, I haven't heard you lay this out like in, in such a way, so I'm really excited to hear kind of your, how your thought process has moved along, but also how you see this working in, in, you know, um, let's leave the, let's leave the small startup behind and let's say, let's just pick a, let's say we have 1,000 engineers inside of a company. How does scalability work from a liaison perspective? Because it sounds like you still have the security team doing more than perhaps I would prescribe in what I think of as a champion. So, if I have 1,000 engineers, in your mind, how big is the security team and what's the role and scope of the security team in working with the liaisons?

23:48Izar Tarandach So here, here it is. It's not as much the number of engineers as the number of engineering teams or projects or units of work or whatever it is. My point is that security team, one of the— again, anecdotal, but I think that there is a body of research to support this. I think that one of the biggest reasons for burnout in security people is a constant context switching. We have to serve all these different projects at the same time. We have to take that stack that has what's the project, what are they building, what could go wrong, who are the contact people, who's responsible for what, who's against, who's for, how much time they give me, is the product manager somebody who I can sell security or not, all those, those variables that define my interface with that project. I have to 3 times a day, 4 times a day change that stack in between the projects that I'm serving.

24:48Matt Coles And to add to that, oh, by the way, do I have the skills either as an individual or within the team for working with the technologies involved? Because those are not necessarily fixed at any given point in time, or they may be bleeding edge.

25:02Izar Tarandach And to add to that, where is this team located? I'm in the US, they're in Europe, they're in China. The difference in time between us. I have to get everybody now in a meeting. How do I get one meeting with 10 people, right? The moment that I have that gopher in the team, all that gets removed and I can think about security. I don't have to think about all the trappings around having that line of communication open and well maintained. That moves to them. All that— who is the person in the team who's responsible for the UI? Who's responsible for the backend? Uh, and has that changed this week? All those internals of the team, they become opaque to me because now I have a question about the UI. I have a question about the backend. That person is going to route that to the right answer, to the right person who can answer and bring that to me. So it's not as much as I am giving technical— I am able to cover more technical issues as a security person. I am able to do more because I am doing less in other things that are less related to my job as a security person.

26:17Matt Coles So, so can I challenge you on something here? So we started this with the definition of threat of a security champion to be an influential member, a technical influential member of the organization. Who can help drive security from within.

26:34Izar Tarandach Yep.

26:35Matt Coles But I think I heard the way that you just described how you utilize such an individual, a program manager or a Scrum Master or a team lead, any of them could do that. Potentially even an admin could do some of those things, things like scheduling and whatnot. So, do you really need a technical influential member of the team?

27:00Izar Tarandach That's not a challenge. That's a really, really, really great observation. The point now is, remember, I removed all my expectations of security learning from that person. Now the people who are coming to me to learn security, to be mentored, are people who really, really, really want it. And if I have them in as many teams as I can, that's awesome.

27:23Chris Romeo So I'm still, I'm still, I appreciate the additional context you provided in that, but you didn't answer my ultimate question, which was how many people though? Because from a scalability perspective, we have to test this model to say, is it reasonable? Because we're all about reasonable. So, but how many people on that? Like if you're, so I get your security team members are now focused. They don't have to task switch all the time, but how many for a thousand? If I have a thousand, okay, I'll give you a, I'll extrapolate my, let's say, We have, we believe in 8 people per development team and we have 1,000 engineers. So we have roughly what, um, how many teams? 100, 100 and say we have 100 development teams, make it easy. How many, how many security, how many people do you need on your AppSec team to liaison with 100 teams of 8 people inside?

28:11Izar Tarandach So again, I, I don't have data. I, I wasn't able to put these ideas 100% to work anywhere. I'm looking forward to doing it, but right now I don't have any data to back this up. The only thing that I can offer is, as someone who sat in that place in a security team, I remember that way more than I wanted of my time was spent doing that, that gophering rather than pointing at threats and mitigations and risks and whatnot. So if you have one person for every 10 product teams out there, that person is getting burned out right now. And by doing this, you manage to reduce the load on them by a factor of 3. All of a sudden you have one person for 30, uh, uh, teams.

29:16Matt Coles But, but I mean, is there— you could solve this problem, I think, also through a judicious use of program managers. I'll say program managers as a role type, but that isn't necessarily that within the security team. In other words, if the security resource, the expert, is having trouble scheduling with a team and you have a program manager in the security team who can do that liaison work, you don't need the engineering person, right?

29:43Chris Romeo They'll reach out to the engineering lead or work within their system.

29:49Matt Coles So, it can work from either end. But you're not really solving the problem, are you? Because the other part of Security Champions, I think, is to make businesses, the engineering teams, the business, more self-sufficient, right? So, today, you're talking primarily about how do you facilitate security to do work with the engineering teams, and facilitating conversations and getting a foot in the door, and not how do you elevate the role of security so that they're not pressuring engineering all the time, and engineering can bake stuff in on their own or adapt guidelines or standards to the work that they need to do organically within the team.

30:35Chris Romeo I mean, guardrails and paved roads we've talked about already. Like, that plays into this too.

30:40Izar Tarandach That plays into this a huge, huge, huge way. Okay, now, Matt, to your point of the, the, the manager— true, you can. And I think that even when we were working every day together, we had the opportunity of seeing many very good program managers come in and out of the PSO. It's not an easy job. It's not everyone who can do it well. And it's not everyone who wants to do it that way. Not many program managers that I know dream of spending most of their time looking at a Rolodex and constantly updating it because developers come and go very quick. Teams change very fast. Projects change very fast. So my point is, when we promote threat model, we say that we have to collaborate with the team because they know the product way better than we ever will. So I'm translating that into this and saying people in the team know the dynamics of the team much better than we ever will.

31:51Matt Coles Yeah.

31:52Izar Tarandach Right. So I think that by separating these two roles, the one that serves the business with the security liaison, and the one that, let's be honest, as Chris put so well, serves the people because they improve, they become better engineers by learning more security, and eventually, who knows, becoming security or anything like that. I think that, again, I can't speak from experience because I haven't tried this yet, but my gut feeling is that there is value in there. It lets us focus on both sides of the conversation much more.

32:25Chris Romeo I think there's value in both sides. I think the security liaisons and the security champions, as kind of what you've described versus what I've described, will fit different companies in different ways. There are some engineering cultures that security liaison is the only way it's going to work.

32:41Izar Tarandach Yes.

32:42Chris Romeo But there's also some where we can have a more proactive approach to touching everybody inside the organization. And I wanted to share just a little, little some more anecdota because it does kind of play into your liaison idea here though. So as the Cisco programs evolved, a role was created called a Security Account Manager, a SAM, and each technology group in the beginning was assigned a SAM and they became almost the reverse liaison to what you're describing. But I think this is complementary to the idea that you're, that you're kind of working on. They were responsible for connecting with the executive levels of a particular slice of Cisco because Cisco was technology groups, business units. They were responsible for connecting with them, but also for being a resource for liaison purposes up and down the stack across all the different teams. And so that was an evolutionary progression that we made beyond our security advocates team. Because we started to realize there's more to this than just empowering people, teaching people, and getting them fired up about security. There was also a liaison side that was required now that we ended up doing it differently. And I didn't design that. I'm not— I'm not— I'm just reflecting back on what I saw happen in front of me. But it was very much a— it was a liaison-like relationship that came out of that SAM.

34:17Izar Tarandach Look, at the end of the day, I think that the problem, the main problem that we try to solve here is efficiency, right? We're trying to make the security process efficient. Now, we've been trying for a long time to do it with security champions. The problem is that even though it's a good idea and eventually successful with programs like what you described, it takes a long time to get the return of investment. Until that champion can become somebody that you can say, that guy force multiplies me, he's a mini-me. It's going to take a lot of time and pain.

34:52Chris Romeo Yep.

34:53Izar Tarandach On the other hand, something like a security liaison starts giving you return of investment on day one because the efficiency now, the thing that you, the friction that you're trying to reduce sits in the security team. And now you're giving them more time and a bit of a calmer environment to work on.

35:17Matt Coles Why can't it be— why can't that be a dual role though? Why can't the security person, the security liaison also be the champion? So, they start with the basics and then as they grow in capability— and by the way, I appreciate what you're saying about the mini-me. The question is, when do they become equivalent to you? Because that's where you reach maximum efficiency, right?

35:45Izar Tarandach The moment that you turn to them and say, "We have an open position on the security team. Are you interested in joining us?" And they say, "No, I want to stay in development." I have never heard that.

35:56Matt Coles I know you haven't, but that would be peak efficiency, right? Because they take those skills and leave them with an engineer, right? and they become basically security within the group.

36:06Izar Tarandach Yep. Yep. I think that both of us remember a certain, what was he? He was an engineering manager that worked with us, more with you in a certain product. And over time, they started developing security testing by themselves. They started asking questions. They started answering questions until one day they said, I'm leaving this place and I'm becoming a security professional. And we told them, good luck on your way, and, uh, here, join our Slack. That's, that's a great, that's a great ending, but it's one of those, what happens if we train them and they leave, or what happens if we don't and they stay?

36:48Matt Coles Yeah, well, I guess that's, that is that issue.

36:52Chris Romeo Richard Branson.

36:54Matt Coles Yeah, there's, right, well, there's that, but also, I mean, that you, you want to train them if they leave, right, that's on the business for failing to meet their needs, right? But you've trained them. You shouldn't train them to stay, right? Ha!

37:13Izar Tarandach Shouldn't you? Because if you say you should train them to leave or not train them to stay—

37:22Matt Coles I guess that's not exactly—

37:24Izar Tarandach No, no. What you're telling me is that there is already value on the fact that you're training them, which I do not dispute. I think that people will learn, will keep learning, and as they learn, they'll be able to apply that. But finally, you got that polished, almost security engineer person. And next thing you know, you are drinking a beer with them because they're leaving the business. Wouldn't you feel a bit cheated?

37:50Chris Romeo I mean, it comes down to how you approach the world though, right? And I think, you know, the three of us, we're always looking for ways to grow people, to mentor people, build them up. I don't take it as an offense when somebody leaves or transitions into becoming a security person. Like, it's—

38:09Izar Tarandach it's—

38:09Chris Romeo I'm happy for them, and, and it's— it means I did my job. As I— if I'm a senior member of the team, I should be looking to invest into people, grow them, mentor them, and hopefully they get— they— hopefully they go to bigger things in the company than I ever did.

38:23Izar Tarandach But Chris, that, that's, that's because we are Although there is a stereotypical security person and we spoke about that in the past, we try to be good people. So we connect to people on the people level. And when somebody comes to us and says, I want to grow into us, I consider going into a security function growth. We want to support that growth. We want those people to be the best that they can. But at the same time, we have to be honest and say we are in business. And the business gave us a mandate to train people for security so that those people could serve as security people in the business, not in the That's business. So, we are doing a great job as a person, but not as business people.

39:03Chris Romeo The way I always describe it, I have a bit of a jokey way of describing it to people. I'll be talking to a developer and I'll say, once again, selling them on the benefit of becoming a champion, like, hey, you're gonna learn all these new things. I think it's gonna make you a better developer at the end of the day. And I expect you're gonna work here for 100 years, but just in case you don't, and they always do the same thing. It always gets a chuckle. But it allows me to communicate a subtle point that people move around in jobs. Like, I can't advocate, like, I hope you leave, but it's a reality. It's, it's, if you grow your skill, a lot of times in, in big companies, and we're going off on a whole other tangent, we got to wrap this thing up. But you, if you start as a junior member of staff, you can, you never get promoted and salary bumps equivalent to as if you joined the company as a senior.

39:54Izar Tarandach Engineer.

39:55Chris Romeo They just never— I haven't seen a company yet that does it. And so you have to leave to take that, to, to get to an equal level based on your experience. And it's terrible that corporate America can't find a way to keep up with people, because you do end up investing in these people as champions, as liaisons. You're pouring into them, you're mentoring them, and then like you said, you're enjoying a beer at their going away party. But it's really not their fault or your fault. It's the organization's fault for not having a way to keep them at a good salary level that's equivalent to how they've grown. And so we've just identified one of the biggest problems in HR. And so welcome to the HR podcast. We're shifting gears now. We're a whole new podcast about HR.

40:39Matt Coles So I want to raise two points, but in the interest of time, I'm going to only raise one and we'll table the second one for maybe a later episode. Bringing it back to what we were talking about last week, right? How do you, how do you, how do you not get engineering to hate security? Here's the question I want to pose to both of you. If you identify a member of an engineering team, whether they're a junior engineer or a senior engineer and, you know, influential within the group, and they learn security and they take on that responsibility, they graduate— we'll say graduate from champion to a security leader within the organization. Because they were a developer and now they've become security, how do we keep them from being hated by their own team? Wow.

41:31Izar Tarandach Okay.

41:32Chris Romeo Okay. Okay.

41:33Izar Tarandach We have come full circle. Yeah, yeah, yeah, yeah, yeah. So, so, so real story. Real story. Okay.

41:38Matt Coles Okay.

41:39Izar Tarandach Had this guy defined as a security champion. Every sprint, he would wait until the end, raise his hand, and say, "Okay, can we talk about security now?" That lasted for a month. After a month, it was, "Okay, are we at the level now that Johnny Ploney will just say, 'Can we talk about security now?'" And I saw that team turn into like, "You're not one of us anymore." You are now the security guy. Here, I wrote the line of code. Can you come and read it and make sure that it's secure? So, yeah, it's a thing. It's a thing. We may be alienating people when we elevate them to that— sorry, not elevate, but when we put them into that role.

42:28Matt Coles We categorize them.

42:29Chris Romeo Yeah. And reclassify them.

42:31Izar Tarandach And it's a true thing. We are— what's the term? Authorizing them. Because we are making them all of a sudden developer plus.

42:38Chris Romeo Yeah, that's true. That's true. That is something to think about. Let's wrap up our first conversation we've ever had about champions. I'm sure we'll have some more in the future. I want to plug a couple of things. We mentioned Dustin, I think. If we haven't, Dustin Laird, he's written Security Champion Success Guide. Check that out. You can just Google for it. It's a whole site. He's put a bunch of his thoughts into it, into what it takes to build a champions program. I've also released the Security Champion Framework. Isar has already commented on it, but that's out there on GitHub, Creative Commons license. So you can go tear it up, throw it away, add to it if you want, whatever you want to do. But it's another couple of resources out there as you're thinking about champions programs. And once again, thanks for joining us on the Security Table and the HR Podcast, which could be coming soon.

43:26Izar Tarandach And people, it's getting cold and silent around here. So if you have something to say, please let us know. We have so many places where you can go and write your opinions and your ideas and question us and tell us that we have no idea what we're talking about. And we would love to have your input as well. So please go there and do that.

43:42Chris Romeo And that's a great way to leave it right there. Thanks, folks.

Transcript supplied by AssemblyAI. It may contain transcription errors.

View the original transcript on Buzzsprout

Originally published as The Security Table. Part of the AI Security Table archive.