Episode 7

You Can't Vibe Code a Security Startup

with Jeremy Philip Galen of Charlemagne Labs

Show Notes

My guest today is Jeremy Galen, founder of Charlemagne Labs. Jeremy spent twelve years at Meta working in privacy, safety, and security — most recently five years as a product manager in trust and safety, focused on machine-learning content enforcement, account access, impersonation, and plagiarism.

He left to start Charlemagne Labs, a New York startup building what he calls a "digital bodyguard" — an on-device AI assistant, Agent Charley, that steps in before a worker clicks a dangerous link or pastes sensitive data into a chatbot.

The company's research recently landed in Meta's safety report for its frontier model, Muse Spark, where Charlemagne's benchmark measured how capable leading AI models are at multi-turn social engineering. His core argument is that the old "think before you click" model of security is broken, and that risky digital behavior should be treated less like a moral failure and more like a public-health and system-design problem.

Key Highlights

  • Selling consumer security software is a non-viable market because consumers buy what they want, while businesses buy what they need.
  • The open internet operates as an active battlefield where users face direct threat vectors from sophisticated foreign adversaries.
  • Falling for social engineering scams is entirely situational, rather than a reflection of an individual's intelligence.
  • Real-time, automated AI interventions are far more effective at enforcing digital hygiene than relying on static digital literacy training.
  • Over 90% of modern cybersecurity incidents originate from human risk vectors where an individual is directly targeted or manipulated.

Chapter Timestamps

  • 00:00Introduction and Guest Background
  • 01:02Career Transition and Startup Journey
  • 02:33Consumer vs. Business Security Market Analysis
  • 03:56Personal Motivation and Scam Prevalence
  • 05:09Social Engineering Sophistication and Victim Blaming
  • 08:01Big Tech vs. Startup Challenges
  • 13:59Fundraising Reality and Survivor Bias
  • 18:05Digital Hygiene and AI-Powered Protection
  • 22:06Privacy-First Architecture and Local Models
  • 28:18Democratizing Security and Luxury Concerns
  • 31:59Meta Collaboration and Industry Standards
  • 35:16Founder Advice and Problem Selection
  • 38:08Company Information and Target Market

Resources & Links

  • Learn more about Jeremy and the company at charlemagnelabs.ai
  • Listeners who sign up for the Pro plan can get 6 months for free if they use the promo code ROB2026.
  • Rob Leathern

Transcript

There may be transcription errors: we apologize for those in advance.

Rob: Hey everyone, welcome to Won't Fix. I'm Rob Leathern. Today I am talking to Jeremy Galen. He's the founder of Charlemagne Labs, a New York City-based startup that is building AI-powered defenses against social engineering. And before the company, he spent 12 years at Meta working on privacy, safety, and security. The last five as a product manager in trust and safety, working on machine learning, content, enforcement, access, impersonation, and plagiarism. His company recently had its social engineering benchmark featured in Meta's official safety report for its MuseSpark AI model. I think you'll enjoy the conversation. We talked a little bit about starting companies and how different that is from working in big companies. So please stick around. And also next week, we'll be hearing from someone who works at Google about some very interesting things that they've been building to keep people safer. So excited for that as well. Let's hear from Jeremy.

Rob: How are things going? So it's been— it hasn't been that long since we caught up, but what have you been doing? Just give it— walk us backwards through what you've been doing for the last little while and let's start there.

Jeremy: You got it. Yeah. So we overlapped at Facebook, now Meta. I was there for 12 years and had three different functions there. I was product marketer, and I worked in public policy in safety, and then became a product manager for my last five years. I left last year to work on a startup I've co-founded and now the CEO of Charlemagne Labs. It's named after my dog, Charlemagne. Our product is called Agent Charlie, which is also named after the dog. She goes by Charlie. Yeah, the idea is like a watchdog. That's why I think it'd be amazing if everybody, primarily employees of companies, had a security watchdog.

Rob: Why do you say employees, employees? Like, is that— tell me more about that part of it?

Jeremy: Yeah, so, you know, I did a lot of consumer tech, obviously, at Facebook, and then I actually worked on Integrity for Workplace in London for a year. But I say employees because selling consumer privacy or security software in 2026 is not advisable. If anybody listening has questions about why I say that, happy to give you the story of the first few months of my startup journey where I thought I would be able to take down LifeLock. And I just don't think you can sell it. It's like not something people, you know— they say consumers buy what they want. Businesses buy what they need. Businesses need security for their employees. Consumers don't wake up wanting a higher, a better security posture.

Rob: That's really interesting. So you're saying that the consumer market for security software is like a null operation. It's like not a thing. It's a don't fix, won't fix. Sorry. It's a won't fix.

Jeremy: I mean, I think the sad truth is like the government should do it with different aspects. And I'm excited to get into it with you. But I think that it's an absolute abject tragedy that we are able to have so much open communication with foreign adversaries and foreign actors and threat actors. And if you look at all of the offline analogies, like, you know, you just wouldn't be comfortable living in a society paying taxes where your local municipal, state, or federal government wasn't doing the basic to keep harm out of your marketplace. You go to the— go downtown and buy something, you rest assured that criminals trying to hurt you have been deterred by policing policy or are already in jail. And the open internet is open and includes harmful actors. And we're on a battlefield. Actually, one of the reasons I got into the space— I worked on adjacent things at Facebook over many years in different teams, but my mom was attacked on a vishing scheme, a voice phishing scheme that was a landline attack. And she lives in rural Maine and didn't succumb financially to it. But the experience of being almost injured led her to cut herself off from the internet. And it's certainly harmed our family. And yeah, I mean, the financial injury would significantly have harmed us. This harmed us less, but it still has cut her off from the modern news and communication ecosystem. So it hits home. I don't know anybody that doesn't have a scam story. Either you haven't fallen for it and you were like, I almost fell for it, or you know a neighbor or a friend or a relative who has been injured. So it's omnipresent and it's profoundly unfortunate. And it's a modern scourge that AI is only making worse.

Rob: Yeah, I agree with that. By the way, so last week I was in Lisbon at the Global Anti-Scam Alliance conference. And one of the things they did was, you know, the first session, people sat down in a keynote and, you know, they basically said, well, you know, there's— I know over 100 of you already have been scammed just this morning, even though you probably think of yourselves as pretty sophisticated. Because what they did is they had someone standing just before the registration desks with a QR code, you know, promising accelerated access, accelerated, you know, getting your badge or whatever it was. And it was basically like a third party or someone obviously acting on behalf of the show.

Jeremy: A simulation is effective. That proves a point. Wow, that's clear.

Rob: I was covering it. It was smart. And so I think a lot of what they were— and in that case, it was essentially— a lot of it is so situational, right? And I told the story—

Jeremy: Absolutely.

Rob: I told the story before, which was like, I have some very sophisticated friends who don't seem like they would get scammed, but just the right confluence of factors and it happens.

Jeremy: Exactly. Context, your guard is down because you feel secure. You're at a security conference. The urgency of getting through the line. I mean, it has the classic ingredients. I think it has nothing to do with IQ whatsoever. The fundamentals here are that you are a victim. And one of the great damages we do is refer to people as having made a mistake in the kind of, you know— oh, that employee made an— this is human error or human weakness. I think that linguistic— the sort of sociolinguistic framework there is a huge disservice because it implies that you could have done differently, but you're actually a victim of a very sophisticated adversary who's using everything at their disposal to do this one— their job, which is to harm you. It's very unfortunate that we have that mindset of— yeah, I have a physician friend who told me that they have HRM— human risk management— at the hospital. And he said that if you click, you know, if you click on the simulation emails more than once, you actually have an HR meeting where you're at risk of disciplinary action. And I'm thinking— I have said this before, but it's like, what the hell are we doing? Taking a doctor out of medical care and treating them like they did something wrong by being tricked. It's like— it's entrapment. It's like when the cops try to sell you drugs and bust you. It's like, what are you— what are we doing? You know, and is that going to change a doc— a doctor's responding to urgent need. If somebody writes an email that's urgent and looks compelling, like— we're in a bad way. We need a new paradigm and fresh systems design.

Rob: So tell me a little bit more about your startup journey. I think— let's talk about— what I want to talk about the product and your insights and whatever. But like, I always find it fascinating how people— like people who've been at big companies and then gone startup or vice versa. Like, I think it's always fascinating how your perception of these different shifts are. What's hard about one and easy about the other or vice versa. So tell me a little bit more about the journey and the transition.

Jeremy: Great question. You know, it would take the whole 20 minutes to give you the download, but I would say that doing trust and safety work at a large company is challenging because it isn't— unless the company is itself a security or safety company, there's friction. I mean, there's— Facebook was famous for having— I joined in 2013. It's hardly the early days, but it still had a taste of a startup. And it was famous for having friction, but it felt productive. I think the later years that I was there, it was just really hard to get the job done because there were business interests that collide directly with trying to make different products or surfaces more safe, which is completely understandable and is a story that should be told. There are trade-offs that commercial platforms face. I also think they're dual moral commitments. And I've actually been trying to get this gospel out there about— it doesn't exculpate Facebook as an actor in any one field. But when you try to make something private, you're going to make it inevitably less safe. And I think of it as a trilemma with free speech being the third option. So you could kind of go all the way in on free speech, but you'd have a lot of safety issues and potentially privacy challenges. Go all the way in on privacy and you'd have a lot of safety problems and maybe you could see how the trilemma works. And Facebook in different surfaces— Messenger, Marketplace, dating, et cetera— has multiple moral commitments. If it's a global company that's serving everybody, including advertisers. And so I would say that that complexity of trying to get a job done is paired with the scale of impact. So I worked on shipping Facebook Dating when I was in public policy. It felt really impactful to be able to make it safer. I work in product safety on the global safety team. That's amazing. Like, you know, as a startup, I don't get to have that kind of impact on people's lives because I don't have a user base, a significant user base. So it's a different— it's a very different job. So that's number one for sure. Then number two is— I feel like there's the— like a friend put it to me this way, it's like a lot easier when someone puts food in front of your face and you just eat it than having to go out and find the food. And so as a non-technical founder, I didn't expect it to be as easy as it is for you to raise money, but I expected to leave Facebook and be like, I was there for 12 years. I built a team in trust and safety. I worked on privacy. Like, where's my money, Sand Hill Road? And like, there wasn't easy money. You know, I found it extremely hard to— I needed to raise to get the co-founder. You know, I needed the co-founder to get the raise. It's just the beginning of these endless loops. You know, you need your first customer to get your second customer, but you need testimonial to get your first customer. Every one of these— so startups face exactly the opposite challenge. Like Facebook, you can do incredibly powerful user studies, A-B testing, different designs and trade-offs and come to like really powerful insights. You can't do that with a startup. You should just ship and hope it works. And, you know, so I always thought it would be easier because I suffer from a sense of personal exception. I thought everything would be easy. Rules don't apply to me. I'll do B2B and B2C. So it's hard. And I found some investors that believed in me that I'm super grateful for. And I'll give them a shout out on the pod. Chris Howard, Rafael Corrales, and Kevin Carter. They gave me a shot. And with their support, I was able to find a co-founder. Crazy contingency. I think Facebook or a large company, there's an org chart. If you need to get something done, you find a person responsible for it. Find a co-founder in 2026. Everyone's telling me, why do you need a co-founder? Just vibe it. And I'm like, the fuck are you talking about— vibe code a security startup? Who would buy that software? I need a security expert who's going to stand by the security of our system. And the contingency, I would say, is the primary principle I sort of live by here, which is that you have to be open to the possibility or create the conditions for the possibility of the serendipity that leads to the next thing. So you really don't have— there's no playbook. I found my co-founder in a WhatsApp group for AI-related tech in New York City. Random thing— I would, you know, noisy as hell. I muted it six months ago. But Dan— like, what would I be? I'd be nowhere without my co— I'd be somewhere else. You know, I'd be in a different life path, like sliding doors. I found a head of research who's a brilliant ML consultant and super collaborator. He's married to someone I went to college with. And I was at my 20th college reunion last year and told some folks that I was up to some AI and safety stuff. And the word— word didn't spread, but like someone else heard that and said, oh, I talked to someone who's also doing— so after reunion, I saw in my spam folder in LinkedIn, like the other box, someone I'm not connected to say, oh, my husband is looking for some opportunities and you may have interests that overlap. Look at those contingencies. If I had skipped my reunion, if I'd not checked my spam box, if I— you know— and now, so that I would just say is the number— those are the number one and two things. It's just like, it's easier to get stuff done and it's— it's also full of friction. And then being a founder has been extremely hard to get things done. And, you know, you rely on contingency.

Rob: Yeah, the first startup I started, I met with so many people on Sand Hill Road who gave me— you know, were like, oh, great to meet you. And like, there's like zero chance for doing— like, the best— actually, the best answer I got was something about, you know— because it was an ad startup, right? So it was like, there's a lot of gold— this was a quote from the VC— there's a lot of gold where you are, but there's a lot of people digging. We don't think you're going to find any, you know? I mean, I don't know, you know, so— but I think it can feel like fundraising, especially, I think, is the thing that is like— feels to me like the most alien for founders. It's like— just as like, it's a hard slog. It doesn't actually matter. I think it doesn't matter what you've done. I mean, sometimes you're just like, okay, sure. If you're like one of these— there's a certain set of people where like, okay, they're gonna, you know, it's gonna take them five minutes. I talked to one founder where basically, like, you know, he got a term sheet before he even had a deck. I mean, like, sure, that happens. But like for most of us, that's not happening. I think it's survivor bias where— or whatever, some sort of selection bias where you, you know, basically you just hear about these people with $20 million, $200 million, these raises, and you're just sort of— you read the headline and you're like, everybody's raising. It must be so easy. You have no idea what they went through to get there. And you certainly don't see the people. No one's ever written a TechCrunch article about the guy who left large tech, tried, tried, tried, and didn't raise. And it's not a story, but there must be a lot of those. And there are millions of people out there founding companies.

Jeremy: I mean, I would definitely say that if I had advice, if I could share with all the humility that I can muster, it's like— you just need one. It's like dating, you know— or, you know, some people date more than one person, but that's the basic idea. It's like— you know, if you're finding that things aren't working, it's really easy to interpret it as I'll never succeed. Catastrophizing is extremely tempting. The fact that I got a no from Rob must mean that nobody will ever say yes. And logically, rationally, that's just not true. I mean, it doesn't obtain— like no, Rob's no might mean a million things. It's like the same thing with like not getting emails back. And that's the other thing is like— if someone didn't reply to an email in a corporate context, it was like you went to their manager and somebody replied, but you pushed in— somebody. But people don't reply. VCs don't follow up. Customers, you know— sales is the other context that's pretty foreign unless you're experienced in sales. Like I'm doing a lot of sales right now and it's frustrating that people's timelines are not the same as my time. My timeline is let's go. The adversary is equipped with a superpower. It's only getting more dangerous. If my solution works, why don't I have— I want five new customers a day. What the hell is that? Like just get in the party. And it's not that— doesn't work that way with B2B security sales, at least in my experience. And, you know, I always have to remind myself I'm not hearing back from somebody because it may just be a maybe, or they just don't want to think about this right now. But it doesn't mean they won't work one day. I do know that if somebody really wants something, they know how to buy it, right? It's obvious. If you really want something that somebody's selling, it's not confusing. Like there is a product that I could tell you right now I'm making and you could say, damn, I need that right now, you know, and you would buy it. That's, I think, one of the hardest things is like— I have nothing to compare this to. So I believe I have all the founders' conviction in the world. I believe in our product and the team. I believe in the startup hypothesis, but it's still extremely early. And many folks I talk to are like, great idea, should exist. And I'm kind of like, well, why aren't you buying it? You know, and there's no easy answer. It's not— you never know what's in someone else's thing. They don't have the budget. They already have a stack they believe in. They're leaving the job in a week anyway. It's just— you never know. And so there's a lot of uncertainty.

Rob: Well, so it's a good segue. So tell me more about— you mentioned to me before employee digital hygiene or the digital bodyguard. Can you tell me more about that and the product and what you're building, what you've built?

Jeremy: Yeah, exactly. So I think the premise is that we want to use privacy-aware or local AI to prevent employees from making mistakes. The kind of— here I go again, calling it an error. I told you it's not an error. But for simplicity, it's a mistake. You click on a link that is harmful. You're being attacked. So that's the biggest mistake you can make as an employee. The second biggest mistake is putting data in the wrong place and jeopardizing customer or business information. And the idea that we're working with is that— and this is sort of the digital hygiene idea— there's a really cool blog post from a year ago, 2025, May— Andrej Karpathy— but he wrote a post about his personal digital hygiene. I don't think he's the first person to use the phrase digital hygiene. And it really alerted me to the idea that unless you're an expert, like he is— his post is incredibly detailed. There are 50 things he does that I don't do, you know— from simple things like VPN to very complicated password management and vault, secure file transfer, et cetera. So he's the gold standard, let's say. And my premise is that it doesn't matter if you're aware— like education and digital literacy aren't relevant in a world where adversaries can just churn out simulacra of legitimate communications. If it's 100% verisimilar, you know— set aside deep fakes, which I'm sure— I know you've talked to the other experts on the podcast. But in my case, it's like— if you just get an email, or a message on Messenger, WhatsApp, or LinkedIn that is so similar to what it would sound like if Rob reached out to me, it's just not relevant to refer to literacy as my problem. Like, illiteracy doesn't mean I'll succumb and literacy doesn't mean I'm going to have my guard up. So hygiene is a better analogy, which is like, you know, I need to just maintain the best possible standards. Like— here's a bar of soap, Rob. Take a fucking shower. You know, it's like every morning you brush your teeth. Like, someone needs to make you do this. And the idea of the agent is that it would intervene and do it for you. So if you— you know, I believe in a future where a sort of private bodyguard would enforce your hygiene. And just from like— the companies have an acceptable use policy that might involve your use of agents. It might involve your interaction with the outside world or new forms of communication and lots of threat vectors. And it just makes— to me, it makes a ton of sense that I'd have— like, I have Grammarly that helps me write. I don't actually need to know the difference between syntax, diction, grammar, spelling. I mean, I don't even know the difference. I don't know the difference between grammar and syntax. I forget. I knew once when I had this, you know, and I went to high school. But the reality is I don't need to know that stuff because the little Grammarly wizard just kind of comes in and offers me suggestions and I decide if it sounds better. So I think that kind of real-time, proactive, assistive model is what— how it should work in the future for employee security.

Rob: Yeah, and you mentioned obviously the privacy side of this, which is interesting, right? So, you know, some of us may have worked at companies or know of companies where they're logging every keystroke, not really because they want to see what you're writing, if you're typing into a personal Gmail box, but they want to see things like if you're typing your— pass— or corporate password into some other unauthorized service. But that's obviously a big part of this, which is like, how do you make sure that you're also not unwittingly creating other threats that might be less obvious?

Jeremy: Or alienating employees. So I think of the privacy in this context— there are two layers. One is the employee privacy. So what is the employee end user's experience of data use and deletion, et cetera, in the local context? And then what is the company's perimeter privacy with respect to the outside world and including large language models, foundational models? And I had a different view on this a year ago. I did some customer discovery when I was still looking for a co-founder. And I had a very impactful half-hour call with the IT head of security of a small video game development shop, like 50-person distributed workforce. My initial idea was to— it was a very expensive idea. Looking at tokens now, I'm extremely glad I didn't do this. The initial idea was to actually take a screenshot, sort of OCR, a real-time view of what you see and try and figure out how to detect and alert about risk. And this IT buyer said, there's no way in hell I'm going to let you take a screenshot of everything my employees are doing and send it to OpenAI or Google or Grok or whatever. And it got me thinking a lot about local models, which is the architecture we went with. And so we ended up fine-tuning Gemma 3, the open source Google small language model. It's 270 million parameters. And so I was thinking a lot about employee privacy as a value prop. And I've shifted that and learned from trying to sell. The security buyer doesn't care about employee privacy. I don't know a single security purchaser who says, oh, I'm so glad that you're protecting the privacy of the employee. Their job is the privacy, the perimeter, security of the company, its network, its data. So a very related value prop that is working is this idea that, you know— it's a headline, sub-headline of our— it's H2 on our website— is like, we can keep your company safe, Rob, without any of your business data touching an LLM. And that's a powerful— that's more powerful because there is concern, even though people have signed the business agreements with apps. Like, let's say there's this AI, this ChatGPT wrapper, and they say, oh, we promise you none of your data will be used in training and validation. They're relying on an agreement with OpenAI and Anthropic, which— I'm not accusing anybody of anything, but I've seen— these companies are going public. And when they face shareholder pressure, I wouldn't be surprised if in the next year— I mean, everyone says, oh, the whole world will melt down if they change their data practices. I've seen this movie before, and the whole world doesn't melt down. What happens is an incremental change to those policies that benefits the process, or the data crossing. In this case, it may not be training validation, but it might be ads targeting. And all of a sudden, I've been using this Gmail inbox organizer that just wrapped ChatGPT. I've been paying nine bucks a month for this thing. And unbeknownst to me, every Gmail I have is part of some aggregated data set that isn't really what I control. And that's going to be an increasing concern with an LLM-powered world, I predict.

Rob: No, I think you're right. I was actually having a totally unrelated conversation with someone on Threads, which was— they pointed out that when you use the Shopify Shop app, you're giving— in order to do package tracking, you are giving Shopify access to all of your emails. And that's all of your emails. Obviously, it doesn't know a priori necessarily which are the shopping emails. And people say, well, this is why Amazon took order details out of their emails and yada yada yada. But anyway, regardless, like— the payoff, the value of getting shipping updates seems to not be worthwhile the risk of every single email that you've ever received being read by—

Jeremy: Exactly. And you could tell a story that's like, well, this is how we got to where we are today with data brokers and all of our data. We've created a large digital footprint for ourselves and practices change. Companies are bought and sold. And the truth is, there's really no— the best way to be is to be private. That's what privacy by design dictates. So, for example, with our small language model, Agent Charlie is a locally hosted small language model. There's a sort of tiered approach where if a company said, you know, we really don't want telemetry, we can accommodate a world where it's very limited egress from the small language model. We have a browser extension that makes the application work. And there's an extraction of the signal that we use to process the URLs in close, very close to real time. And so if a company says, no, I want that telemetry, I want to know what Rob's doing in his browser, we can also— they're the buyer. We can accommodate that. So I feel as though it's important to offer the choice. But I'm glad that we started from a place of parsimony because then I could sell to the government a completely air-gapped version of this, which I'm not trying to do. But if anybody's listening and knows how to sell to the government in under 24 months, I would love that. But any government, by the way— it doesn't have to be the U.S. government. But I could see a government being really attracted by a private solution that offered security in a local fashion because that's very compelling when you care about privacy as much as one cares about security.

Rob: I think one of the things that is interesting when you were talking about agents— I actually like the bodyguard metaphor framing because that makes kind of sense. Although I also worry— I mean, again, this probably doesn't necessarily apply as much to your business-facing use case, but like I worry a lot about safety products becoming like a luxury good, or becoming something where— earlier in the conversation you said, well, maybe the government will have one thing that everyone gets and then the wealthy will be more protected. And again, in some ways, this is how society works today. But I feel like hopefully we can have things where more people benefit from technology and from safety technologies than historically perhaps might have been the case. So that's the interesting thing on the bodyguard framing.

Jeremy: Yeah, no, it's super interesting. I share that moral mission, and the anxiety that it won't be achievable. One actually cool thing about using an SLM is that we don't have any cost of goods sold. We don't have any COGS in the same way that others have token costs. So I have talked about Bombas socks. There's like a buy one, one donated— one purchase, one donated model. TOMS shoes did that. I've thought a lot about ways— and I've tried to market this and it hasn't worked— but it would make a lot of sense to me that you might make it free for everyone over 59, or retirees could have it for free, or teens could have it for free. And you could charge a separate population. But in the B2B context, that's not as relevant. It's funny you say that the bodyguards are expensive. I certainly don't have a bodyguard. Nobody wants my body, so I don't need anybody to guard it. But the idea of— what I've always wondered, what does Bill Gates have? Imagine his physical security. It's kind of like the president. But the digital cybersecurity that he has must be a proper SOC. He has a full-time security operations center with inbound and flight tracking and context awareness for all of his travel and who knows. And so— in a way, the early, early pitch I started was to think about that and say, well, what if agents could give everybody that? And so I actually started with a democratizing idea, which was like, you know, why don't we already have agentic SOC analysts and that's a thing? Like, why doesn't everybody just have an agentic SOC? And so I think— I mean, somebody will make that happen with consumers. I don't know which actor— could it be a telco? Would it be— is Apple going to put it on, you know, in nano models onto local devices eventually? Is safety going to be the first thing they do? I don't know. You know, my guess is that they're going to try to win the productivity battle, the Siri battle. And I think we've seen— both of us have seen this. It's like when I was trying to get— when LLMs were just becoming a thing and I had cooked up an idea in my own job at Facebook to use new technology— it was like, do you think anybody's going to be throwing GPUs at a PM in safety? Like, no. I mean, the goal, you know, the focus— the priority was really on all the other things that needed to do with the new technology. So it's hard to see even Google. I mean, Google has 3 billion users in Chrome, including business users. And they've shoved some nano model in there. And, you know, I know there must be one or two PMs at Google right now trying to do all the safety work they possibly can with limited allocation of compute or whatever. So yeah, it's just hard to see safety and security being the first priority. And that's unfortunate.

Rob: Funny, you should mention that. We're actually— the next guest on the pod is going to be from Google talking about some of the stuff you just talked about. So I guess we'll find out. Next time.

Jeremy: Yeah, that's great. Tune in. Keep tuning in. Like and subscribe.

Rob: Yeah, like and subscribe. Tell me— so speaking of big companies, coming back full circle to that topic, you folks have done some stuff with Meta. Can you talk a little bit about that?

Jeremy: Absolutely. Yeah. So I never worked with MSL and Meta Superintelligence when I was at Meta, but about seven or eight months after founding Charlemagne Labs— and I called it Labs because everybody does that with their startup name now. I don't know why. So I was like, if everybody's doing it, I got it. But I genuinely thought that we'd be researching more of the LLM capability in the defense side, which is obviously what we've done. So the idea would be like threat detection— like that would be the laboratory. And what ended up happening was the Meta team that's responsible for the security testing for— now it's called MuseSpark, but it was unreleased at the time— they got in touch about us helping to build an eval suite to assess the capability changes in the scam and fraud space of the essentially of offense, right? Like how good are models at multi-turn persuasion. So Richard on my team kind of led the research work, got a cybersecurity professor at UVA who's helped as well. And my co-founder and I— what we ended up doing was really cool. We got an eval suite built that is essentially a benchmark for capability in the scam and fraud space. And I found out along the way, it's actually part of the compliance with SB 53, the bill in California that regulates the frontier labs. And it's mildly dismaying that we've been unable— I've tried to reach folks at Google DeepMind and Anthropic and OpenAI about also adopting this benchmark now that it's been used by Meta. It makes a lot of sense to me that it could be used by the other frontier labs. And they've not responded, which is unfortunate because I think we're missing an opportunity to have a standard way of referring to model capability in a vital domain. SB 53 refers to it as theft by false pretense. And I feel as though— if you can let me stand on this soapbox for another 30 seconds— I feel as though we're paying an awful lot of attention to cyber risk in the technical domain, namely exploits and vulnerabilities, the infrastructure and code issues. And that is absolutely the right prioritization. We must secure our infrastructure. But after that, we have the human risk again. And so if we don't address human risk in the same moment, we are going to be back at sort of the status quo ante where today, 90% of cyber incidents begin with a human making a mistake or being targeted.

Rob: So tell me a little bit more about— I know for a fact people have like reached out to me. And so I know for a fact there's people working at Google, Meta, Pinterest, whatever, a bunch of different companies who want to start a company. They want to find— they might be technical, I want to find a co-founder. They might be non-technical or less technical. There's definitely lots of people who've been, you know, who have free AI tokens, at least for now— free tokens to build stuff on the side, you know, at some of these companies. But what's some of the advice you would give folks who are passionate or interested in starting a company? What are some of the things you would do, some of the things you wouldn't do, some of the things you've learned, aside from the fundraising sucks, which obviously we know.

Jeremy: Right. Yeah, I would say pick a problem. I mean, everyone sort of says this, but it bears repeating. I think you want to pick the problem that you know you won't tire of thinking about. I mean, it is just absurd to remind you, but you will be thinking about this in a way— you know, at a large company, you can change teams, you could change functions, you can move cities, you know, you can grow and change in different ways, et cetera. But as a founder, if you pick a problem— like, I'm going to work on long distance education or telemedicine or, you know, you pick that problem— it has to be something you will ride or die with because you will be involved in it in an incredibly immersive way that is hard to describe without being there. It will— it will obsess— you're falling asleep at night thinking about it and dreaming about it, and you wake up and you're sort of thinking about it. And, you know, if you have to pivot, that's fine, but you find that problem because it's kind of the whole thing. You know, you're marrying a problem as a founder. And then I think the second thing— knowing how to manage noise. Like one of the most annoying things, even like friends and family— you tell them, oh, I'm working on this thing. People will naturally say, oh, I just saw an ad for an AI startup that's kind of doing the same thing. And like there's a natural thing— people look for patterns and similarities. And so it's not meant as a— it's not meant to be like, don't do your thing, don't build your thing, you won't succeed. But it takes resilience from that very beginning of just idea phase. You know, investors will be like, I'm conflicted out. I have a portfolio co. And like my dad used to say— Coke and Pepsi, hey, they're two incredibly sugary beverages that seem to sell every day. You know, they taste the same, sort of, you know, but people have feelings about them. So, you know, there's a lot of noise, and dealing with noise is annoying, but that's another thing— like buckle up for that, especially with the lowered, the reduced cost of building. Like if somebody likes your startup, they can vibe code a clone of it in one prompt in Lovable. Are they going to execute? You know, it's like investors ask me about moat and I'm like, there is none. I don't know— what do you want? Like, you know, there just isn't. I'm not a medieval king with a drawbridge. There's just no moat. Sorry. And so you have to keep building until you'll build a business or you won't. But there's some old rules like, oh, I have a provisional patent. You know, good luck defending a provisional patent against someone who wants to do what you're doing. So lots of noise.

Rob: I remember filing a patent many, many years ago, back when business process patents were still a thing. And like, yeah, good luck with that.

Jeremy: Absolutely.

Rob: So where can people find out more about the company? Where can B2B buyers give you the inbound— sales leads for you as opposed the other way around?

Jeremy: Excellent. Thank you. CharlemagneLabs.ai, Charlemagne Labs. I'm Jeremy Philip Galen, and you can find me on LinkedIn. We are very excited about design partners in kind of businesses upper mid-market, meaning the companies of 100, 150 employees that are SOC enabled. They've got a security operations center. Those are the kind of companies that we're partnering with right now that make a lot of sense. It's also really available in a self-serve way to SMBs, small businesses. So we have a free basic plan, completely free for a single user to try Agent Charlie. And yeah, excited for feedback and stay safe out there.

Rob: Yeah, we'll have a link to all of that in the show notes. So I encourage people to check that out. But really want to thank you for your time, Jeremy. It was really great to learn more about what you are up to. And I hope we'll keep hearing more about the company.

Jeremy: Thanks, Rob. Great to be here. Take care.

← All Won't Fix episodes