Episode 15

AI Factcheckers and $3.99(?!) Costco Hot Dogs

with Alexios Mantzarlis of Indicator

Show Notes

Alexios Mantzarlis has spent his career on the unglamorous side of the internet. He founded the Italian fact-checking site Pagella Politica, then led the International Fact-Checking Network, then spent several years at Google building an adversarial red team focused on content risks from generative AI. In 2024 he became the founding director of Cornell Tech's Security, Trust, and Safety Initiative, a role he left in June 2026 to go full-time on Indicator, the independent publication he co-founded with Craig Silverman to investigate digital deception. He now runs it from Rome.

If there's a through-line, it's that the cheapest new technology always finds the oldest scams first. In the past year alone, Alexios has written about a North Korean hiring scam; an AI-generated podcast network pumping out 11,000 episodes a day; an AI bot that became the single largest contributor to Community Notes on X; Grokipedia citing a neo-Nazi forum as a source; face-swap apps in the Apple and Google stores; "cheater buster" websites selling fabricated infidelity reports; TikTok content farms; AI avatars impersonating real journalists; lookalike ticket sites and thousands of ads for nonconsensual nude generators that Meta has kept running despite a year of documentation.

We talked about his latest investigation: roughly 8,000 Reels featuring AI-generated women posing as fired Costco, Aldi, and Home Depot employees, leaking "insider secrets." The videos funneled to survey sites promising a $750 gift card that didn't exist. Alexios and Benjamin Shultz traced the whole thing back to a single affiliate account and to a scam pipeline where AI generates the video, the captions, and the landing pages.

Chapter Timestamps

  • 00:00Introduction
  • 2:20Alexios's path from platform trust and safety to independent reporting
  • 4:36Why public investigations can drive platform action
  • 8:09Fact-checking and Community Notes should reinforce one another
  • 11:33Anatomy of the AI-generated retailer gift-card scam
  • 14:57AI used throughout the fraud pipeline—not only in the videos
  • 17:36Recommendation feeds and reduced source context amplify scam reach
  • 22:34Affiliate IDs reveal the scam's business model and coordination
  • 27:20Responsibility across the infrastructure chain
  • 30:50Journalism, trust-and-safety staffing, and the nudifier-ad problem
  • 33:30Transparency, EU ad data, and pro-speech accountability
  • 35:47AI bots in Community Notes and monetization-driven misinformation
  • 41:29Sources, sustainability, and Indicator's next investigative tools
  • 47:51Closing warning against dismantling trust and safety

Transcript

There may be transcription errors: we apologize for those in advance.

Rob: I'm Rob Leathern. Welcome to Won't Fix, where we talk about AI-driven deception, abuse, and scams — and why they're so hard to stop. In software engineering, the term "won't fix" describes a bug by acknowledging the issue but intentionally leaving it unsolved, because addressing it is too costly, too risky, or not worth the trade-offs.

In a way, today's guest is perfect for Won't Fix. Alexios Mantzarlis has spent his career on the unglamorous side of the internet. He founded the Italian fact-checking site Pagella Politica and then led the International Fact-Checking Network. He spent several years at Google building a red team focused on content risks in generative AI. In 2024, he left to become the founding director of Cornell Tech's Security, Trust, and Safety Initiative — a role he left in June 2026 to go full-time on Indicator, the independent publication he co-founded with former Won't Fix guest Craig Silverman to investigate digital deception. He now runs it from Rome.

If there's a through-line, it's that the cheapest new technology always finds the oldest scams first. In the last year alone, Alexios has written about a North Korean hiring scam, an AI-generated podcast network pumping out 11,000 episodes a day, an AI bot that became the largest single Community Notes contributor on X, Grokipedia citing a neo-Nazi forum as a source, face-swap apps in the Apple and Google app stores, "cheater buster" websites selling fabricated infidelity reports, TikTok content farms, AI avatars impersonating journalists, lookalike ticket sites, and — of course — the thousands of ads for nonconsensual nude generators that Meta has allowed to keep running on the platform.

We talked about his latest investigation: roughly 8,000 Reels featuring AI-generated women posing as fired Costco, Aldi, and Home Depot employees leaking "insider secrets," with the videos funneling to survey sites promising a gift card that didn't exist. Alexios and Benjamin Shultz traced the whole thing back to a single affiliate account, and to a scam pipeline where AI generates the video, the captions, and the landing pages.

Looking forward to this one. Hope you'll give us your feedback and share this with other folks. Please subscribe and let us know how we're doing.

Rob: Alexios, hi. Where are you calling in from today?

Alexios: Hey, Rob. I'm in Rome, Italy, where it is still a million degrees.

Rob: Well, even though it's hot there, I'm in Austin, so it's probably hotter here. And I'm officially jealous — I'm always jealous of anyone who's in Italy. It's good to see you. The first time I met you in person was when you invited me to speak at Cornell Tech's Privacy Day, which was really fun, really interesting.

Alexios: Yeah. Feels like a lifetime ago, but that's what an international move will do to you. It was only maybe three or four months ago. Good bunch, though. Good bunch.

Rob: Tell me a little bit about your recent history and what you've been working on. Indicator has been going for longer, but tell me how it all came together and how you decided to focus on it more recently.

Alexios: Very brief background: I ran a fact-checking website in Italy, then moved to the United States at kind of the peak moment for fact-checking on digital platforms. I was part of the effort to get Facebook — Meta — to do more about this, and then ultimately helped them set up the third-party fact-checking program. I spent five years at Google Trust and Safety and left just as we were throwing everything out of the window to get Gemini running, and I had lots of qualms about that. Then I did some trust and safety teaching at Cornell.

And I've always had a newsletter, because I like to share the stuff I find interesting. It used to be called Faked Up — it actually started inside Google. At some point Craig Silverman, whom you know as well, said, "Hey, I'm leaving ProPublica. Do you want to do something together?" And my view is: if Craig Silverman asks you whether you want to do something together — given that he's been at this longer than all of us and deeper than most of us, and is also the nicest guy — the only answer is hell yes.

Rob: Craig's great. I spoke to him a few months ago on this still-pretty-new podcast, which has been a big learning opportunity. What have you learned from the Indicator experience? You run a newsletter, but you also do paid workshops, and you give people a lot of great free content — you show them how to use OSINT tools and so on.

Alexios: The biggest thing I take away — which was kind of my theory of change in leaving the platforms — is that writing about this stuff publicly does have an impact. Covering platform deception pushes it further up the triage list for decision-makers inside the platform, because they have these infinite inboxes and they're looking for shortcuts on which decision to make. Making noise helps things get fixed.

In two years — a year and a half, even — we've gotten hundreds of thousands of accounts taken down across a range of deception techniques, a lot of them in the scams and fraud area, because we write about it and expose it. So I think there's value in this work, both inside and outside the platforms.

Rob: Have you found the response from different platforms to be similar? Or are certain companies more thin-skinned, or more responsive, when you bring these things up?

Alexios: That's interesting. Perhaps surprisingly — given my past at Google, and given that we have a lot of Google paying subscribers on our newsletter — I find that getting a response from Google on our work is more sporadic than, say, Meta. Meta obviously has a more interesting and in some ways worse track record on some things, but it's far more reliable at actually getting back to us on what we found, and often taking things down.

If I can stereotype a bit, I feel that TikTok tends to be happier to take things down than the US-based platforms, which may have something to do with where it was based and the scrutiny it was facing. But every platform is its own special case, frankly.

Rob: And evolving, too — as their policies evolve, as the tools they use evolve, as their public narrative changes. It's really interesting to watch, and you have a super useful vantage point for it. You also see other people trying to understand what they're doing.

Alexios: I'll die on the hill that at most platforms there are still people trying to fix things and do things right. The question is: (a) can you get to them, and (b) do they have the agency to intervene? Taking things down or blocking content is always easier than proactively changing products and going through all the launch processes that you know — and probably have PTSD about — to actually ship something that has a continuous impact. But it's important work.

Rob: I want to spend time on the gift card exploits you uncovered recently — I found that a very interesting slice to focus on. But first: you've worked on fact-checking, which generated a ton of interest, work, and scrutiny at the platforms a few years ago, and has since been supplanted by Community Notes. I think there's a lot of potential value in Community Notes, but now there's generative-AI-powered fake Community Notes — I don't even know what to call them. What's your take on what happened with fact-checking and where we stand with something like Community Notes?

Alexios: Rob, I'm concerned I'm going to spend the rest of the podcast talking about this.

Rob: We'll time-box this one.

Alexios: I have been one of those sickos obsessed with how we can crowdsource fact-checking, forever. Ten years ago, with a bunch of friends, we launched FactCheckEU, a crowdsourced fact-checking process driven by the idea that professional fact-checkers also need expertise, help, languages, all kinds of things the crowd can bring.

So I think there's value in Community Notes. I was obsessed with Birdwatch, which is what X called it before renaming it. But as with all interventions, and all digital safety interventions, it's a complement. There is no silver bullet. In fact, I was quite nervous in the early stages of the third-party fact-checking program when Meta was trumpeting at all its hearings that this is our answer to fake news. It was an answer, and an important one. And notably — to your point about political controversy — it was a pro-speech intervention. Fact checks were always attached on top of posts; posts were not removed.

The really frustrating thing that's happened in the past year and a half is that Community Notes and professional fact-checkers have been set in opposition, when in fact they can be used together. The crowd needs fact checks to point to sources, to get to things the crowd hasn't reached yet. It's a multiplier effect.

But what happened explicitly with Mark Zuckerberg on January 5th or whatever of 2025 was that he chose to use fact-checkers as a symbol of what the incoming Trump administration disliked most, and sacrificially killed the program in a live video as a sign that Meta was changing its ways. Behind all the rhetoric and politicization, what's really going on is that they killed a sensible intervention that could have coexisted with Community Notes perfectly well.

Rob: One interesting footnote: I've spent a lot of time working on ads over the years, and I kind of wanted fact-checking for ads. But that's a completely separate conversation.

Alexios: Yes — there were significant carve-outs, political speech, et cetera. But you can't get everything.

Rob: Tell us more about this investigation. Whenever I see a headline that mentions Costco, I always wonder whether it's about the $1.50 hot dogs. What's the background on how you found this, and what did you learn when you dug in?

Alexios: First of all, the Costco hot dog does feature in the story. My co-author, Ben Shultz — who is absolutely wonderful, and is probably the only person whose social media feeds are more messed up than mine — noticed that in one of the Reels the hot dog sign behind the Costco employee said $3.99. Which is wonderful, right? It's the little sign that AI still has contextual cues it misses when you're not being careful.

To summarize briefly and then dive in: we found a ton of Reels on Facebook and Instagram — AI-generated videos where these employees were pretending to share the dish, the insider gossip they supposedly know. There are plenty of legitimate accounts that do this kind of thing, discount tips and so on, so there's already an inclination for people to follow this type of content.

What this scam network did was insert a fake gift card link at tip number four, always. That link initially sent people somewhere they'd get phished, and then eventually shifted into a classic affiliate marketing bait-and-switch, where you'd click and it would open an ad-laden page for gambling or poker or whatever else.

What makes this distinct is the scale. These videos got over 100 million views on organic content, Rob. I've seen stuff being boosted to people through ads, but the fact that you could push out all these videos — I think there were about 4,000 of them — and reach 107 million views, for whatever view counts mean in the modern platform ecology, is still a lot in my view for content that was relatively simple.

The other thing is that we really saw AI use throughout the whole pipeline. Not in some revolutionary way — just that AI is baked into bad actors' workflows. They've absorbed the productivity recommendations and gone perhaps faster than the rest of us at baking it into their processes.

Rob: That's an important theme, because a lot of times people think, "Oh, if I just don't believe any videos of celebrities, then I'm good." No — the synthetic nature of what's happening is happening at every different point. Where else did AI, or AI coding, show up in this investigation?

Alexios: Besides the videos: the scripts were templated in a way that would repeat the format but change the wording. Pangram is obviously a tool to use with care, but across a big corpus — I've found even in red-teaming it that it's pretty reliable — all of the captions were AI-generated.

The websites people were sent to for the gift cards were vibe-coded, AI-generated, whatever you want to call it, often using a website called Lovable. Which, however, I will call out as having a pretty active T&S team, at least as far as this goes. They had caught some of these before we reached out, and then actioned on the rest afterwards.

And the profile pictures — maybe we can link them in the notes.

Rob: We'll definitely link them.

Alexios: Usually you'd think there'd be the same image reused, or AI-generated, or scraped, or whatever. What they did instead was ask whatever AI tool they used to create twenty-five different variations of the same blonde woman in a Brooklyn-esque apartment. You can see from the back, right — the images change, the buildings in the window are slightly taller or shorter.

What that does is make it harder for a classifier looking for the same image to detect the other accounts tied to the operation and action them. Similarly with the AI text: instead of copy-pasting and having everything be exactly identical and getting caught easily, you can use AI to slightly tweak everything and keep the variation significant enough that it won't get caught.

None of this is rocket science, but what it allows is an explosion in the number of accounts, efforts, and attempts you can make — and with it, the reach. If you push out 4,000 videos, there's a very long tail that got 100, 200, a few thousand views. But then you get the big ones, and those are what ultimately pay off the entire operation.

So gift card scams, gift card lead gen, phishing, or variations thereof — this is not a new thing. But the scale and the amount of testing is new.

Rob: The other thing that's really new is how responsive a lot of the algorithms are to things that seem to be getting traction. I was just talking to someone recently about how much unconnected content people see as a percentage of their feed on Facebook, TikTok, and other platforms. Can you talk about the testing scale versus the actual scale — how much of that is algorithmic, or other things you can infer from what you saw?

Alexios: First of all — because I reached out — besides Costco, Aldi, Target, all of the potential targets have been targeted by this operation. The spokesperson for Home Depot said that while they've seen gift card scams like these forever, they do think they're increasingly common now. I'll take their word for it. And many of the companies I reached out to have had to contact Meta and other platforms about this before. So it's both new and growing, which are two things we can hold in our heads at once.

I think that's true in general of AI harms. Sorry to go on a bit of a tangent, but there's this hype framing of "this is a revolutionary new harm of AGI taking over everything" — the swarms coordinating on taking down Hugging Face, which, my God, impressive, cool, whatever. But a lot of the existing harms can be same old, same old, but much bigger, and still cause harm.

Your original question, which I sidestepped, was also about algorithmic reach. I don't know whether some of this is self-inflicted by all of us who said "filter bubbles, bad thing" — so in part to respond to that, and in part to copy TikTok, which was dramatically successful, Meta has shifted to content where you don't have a connection to the poster.

It did strike me how many of these accounts, when you open them, are clearly trash accounts. They're not even bothering. They have one-liner bios. Their videos shift from "I left Target" to "I left Aldi" to "I left whatever." They're clearly banking on the fact that nobody is actually following the account because they know it or want to track it — they expect people will get exposed to it regardless.

For me, the fact that an account named simply "Laura" can get 24 million views on its best Reel in the sample is remarkable. This distribution mechanism has made it easier to create throwaway accounts that land the occasional one-off hit.

Rob: One thing I've noticed — true for organic content as well as ads on a bunch of platforms — is that the prominence of the poster's identity has been reduced over time, and so has the prominence of the content category. Sponsored labels are smaller, or in some cases nonexistent. FTC guidelines aside, it just appears that as there's more unconnected content, the name of who posted it has diminished. Some platforms have also made it more permissive to be a paid blue-badge-verified account, or whatever the current nomenclature is. I think that might be playing a role in how people respond.

Alexios: For sure. Digital literacy 101 is: what's the source, and can you see it? If you make that harder and harder, and even Google — which used to care about this stuff — has hidden the "about this result" context behind a hamburger menu, there's a lot of frustration.

I understand it. The product manager is trying to make things as seamless as possible, and adding context slows things down and makes them clunkier. But this matters when it comes to information quality and combating deception on the user side.

Meta just changed its label for AI creators — even the naming keeps changing. And it used to be that on Instagram, if the creator had a song attached, the label indicating an AI creator would cycle after you'd seen the song. So, to your point, it's even harder to find. There's really no reason to make these things — which already don't pop out, and which users may not understand — so much harder to see, other than that they're being forced to do it and will do it in the least obtrusive way possible.

Rob: Another interesting thing about your piece: when I casually look at scams, I always ask how it's trying to monetize. What's the business model? What is it trying to get me to do? Is that a normal channel? Sure, you want me to pay my toll charge — but Zelle probably isn't a regular way for your municipality to collect that. So tell me about the business model here. Was that part of how you found this?

Alexios: Follow the money remains the cardinal instrument of investigating anything, online as elsewhere. We were trying to figure out how they were making money, and these ads explained it. When you clicked the big survey button to get the gift card, we saw that the redirect link had one of those habitual shorteners with an affiliate ID code next to it, which implied an affiliate marketing scheme.

The people behind the Reels and the websites go to various more-or-less shady networks that try to drive traffic, and they get an ID. The way we were able to establish that this was one network rather than several accounts acting independently was by seeing that the affiliate ID on many of these separate websites was the same. So there was at least some entity — a company, a cohort, whatever — where the money was ultimately flowing to the same place. That was valuable and useful for us.

Rob: What's your general take on affiliates? One of my guests on this podcast has said affiliates are kind of the root of all evil. That's a bit of an exaggeration, but you must see a lot of this. I imagine you have a nuanced take.

Alexios: We're working on something I won't go into too much detail about, but it looks at how the mechanics of the incentive systems behind affiliate links resemble the incentive systems behind viral posts and pay-per-click. Ultimately you get paid per click, so you want to get that click however you can. In its most excessive form, that leads to a full-on fraudulent front that redirects people.

I don't know that affiliates are necessarily worse or different from the current dominant platform ecosystem mentality of pushing up whatever gets viewed more — the view-based or engagement-based model. I think we fundamentally need an evidence-based, grounded counterweight. For a long time, the reason I was drawn to Google as the platform I worked for was partly the model search was built on — still an ad-based model, to be clear — but there was a sense that expertise, authoritativeness, and trustworthiness should be a factor, and a big one, not just reach and popularity.

Rob: One of the things that concerns me about the affiliate model — and to your point, it's replicated in a bunch of places in the platform ecosystem — is the plausible deniability. "We provide the rails, we're not responsible for how people use them." Or, more insidiously: "Here are our policies, and if someone brings a violation to our attention, of course we'll take action." But meanwhile the money is rolling in, and as long as you don't ask too many questions, it keeps rolling in. Once you start asking questions, you find these exploits. I hope it's a temporary state of affairs and we can move past this "I have no idea what's going on in my ecosystem" posture, but I'm not always optimistic.

Alexios: The affiliate link setups in this investigation are almost impossible to trace to someone who'd be responsible for taking them down. I tried to look into linkthem.net and ta5dy, the actual infrastructure, but it's different from when you have a platform or a quote-unquote respectable affiliate program owner.

We're also seeing a lot of affiliate links for AI tools being used by hustlers and side grifters to encourage the generation of really lousy, crappy content. There, at least, the question is: can I go to the AI company and say, "Hey, these affiliates are encouraging people to use your tool in a way that violates your policies. What are you going to do?" I worked in trust and safety at Google and never saw this on YouTube, so I don't know what the affiliate policies were, but I'd expect that some of these companies now have to have a T&S unit that looks at affiliate marketing.

Rob: Absolutely. And your investigation points to something the platforms struggle with — we struggled with it when I worked on this. How deep do you go? How much of the whole chain do you look at? If you're one of the providers, you're looking at a website with a gift card offer and you don't know if the outbound links are malicious. Do you transit the links? Do you go upstream and look at the videos leading to the offer? It's a really interesting question of how much you need to be proactive versus how much you have to be reactive, just given the volume.

Alexios: This is a lesson every new platform learns. And the lesson is usually that you have to be at least a little bit proactive on the biggest harms — health misinformation, full-on fraud and scams, danger — without even getting into the outright illegal stuff like CSAM.

Bitly made headlines when it decided to ingest the bad-URL set from Google some years back. Bitly could have made the point you just made: we're just a bridge. But that bridge gives people credibility — "Oh, I've seen Bitly links before, I'll open it, what's going to happen?" — and then it downloads malware. At that point you're responsible for lending your credibility to a scam. There's a point and a place for the "we're just infrastructure" argument, but our investigation showed that Lovable clearly feels some responsibility to do proactive and reactive review of how its websites are being used.

Rob: I've seen that a few times — folks like Lovable and some of the other newer platforms are very responsive. They're also high volume, because they're getting used a lot, precisely because they're so easy to use. But it's great to hear they took action quickly.

Rob: That's a good segue. There's a discussion among journalists of, "Are we just doing the platforms' job for them? Shouldn't they be doing what we're doing?" Your earlier point was that this does have impact, and sometimes — maybe often is an overstatement, but definitely sometimes — it leads to policy and enforcement changes, and draws attention inside these companies. How do you think about journalism as enforcement?

Alexios: The crude version is: I could be making a ton more money at a platform doing exactly this work. I used to. The platforms have the resources, the means, and the sophistication to do a lot of this already. Part of it is a triaging question — there's just so much.

But I think the answer is: hire more people. Instead, we're in the midst of an ideological pushback against having humans. And it doesn't mean they can't be AI-empowered humans — it's not either/or — but having humans, particularly in trust and safety. There was this vision that T&S isn't conducive or helpful. But what we see is that if you leave your platform too lawless, users complain and regulators bite. So all the platforms maintain T&S functions. They just maintain them at exactly the size they can get away with.

Not to pick on Meta again, but last week I ran a story about how many ads ran for AI nudifiers on the platform this year. We found more than 11,000. And to be clear, I've been doing this audit for several years — it's tens of thousands, over and over again. Some of these accounts are just changing the final two digits of their names and creating new advertiser accounts for the same crap.

So when you sometimes hear from comms folks or execs at the platforms that "this is an incredibly challenging problem" — some of it is, no doubt. But if one idiot in a room with a Claude Max subscription can find 11,000 ads, I would hope a structured unit with all the tools and a lot more visibility inside Meta can find them first.

Rob: When I was working on this from the platform perspective and we shipped the ad library, and a bunch of companies did the same, we knew this was where it would go. But I don't feel there's been the resulting swing back to "okay, we need to get on top of more of this, given the level of transparency there is." To Meta's credit, their ad library is better than a lot of the others, and others are improving. But I agree — it's in plain sight, it's awful, it needs to be fixed and addressed, and it needs to be explained. Not just "we had fewer user reports this quarter than last quarter." That's not enough information for the public.

Alexios: Let me give you a slightly more polemical angle so I can be a bit catty. On the ad library point, there's this stereotype that the EU is a big, bad, censorious organization — which we could dissect further, and there are elements of that — whereas the US is free speech land. But the only reason the ad library provides clean data about reach in the EU is because the EU forces Meta to do so, and the US does not. So we actually have far more information in the EU about what's going on inside platforms.

If you want to make pro-speech moderation decisions, more transparency and more context is the way. Whereas when Meta takes down US ads, they vanish altogether and you don't get this kind of information. I worry about governmental definitions of misinformation and requirements to do this or that. But I really struggle to see how forcing platforms to be more transparent on things like this — and having a floor that keeps rising on what's expected from something like an ad library — is anything but a net positive.

Rob: Tell us more about some of the other investigations you've done. There are the North Korean hiring scams that everyone's talked about, the AI bots contributing to Community Notes — so many interesting things you and Craig have written about this year. We'll include links in the notes. But which have you found most interesting, aside from what we've already covered?

Alexios: We mentioned Community Notes in passing, but to hone in: I find it fascinating to look into what drives someone to be a good Samaritan on a platform — to the point that they'll contribute, correct, and spend time out of their day correcting someone on X, a platform many of them dislike because of the politics of its owner, in the vague hope that it lands. Only about 10% of Community Notes ever see the light of day and get affixed to a post.

It's a fascinating exercise in trying to build the right incentive mechanisms into a platform. But what we found earlier this year is that with the introduction of AI bots into the mix, the bots write in ways more people find palatable, and they write much faster. They're now stably — we have a tracker on the website — more than 50% of all helpful notes shown on X.

So there's a paradox. This mechanism, which was supposed to bring content moderation back into the hands of users, to democratize moderation and take it away from the machine learning classifiers built by the "bad evil engineers" in liberal San Francisco, has come full circle. Now a group of eight unknown developers behind the AI bots dominate these corrective measures. It's a really fascinating space to keep studying.

The other thing I look at a lot is creator revenue and monetization programs — especially TikTok, but also Meta, YouTube, and others. They've introduced the same incentive we were talking about with affiliate links: push out whatever you can get 100,000 views on, so you can collect the RPM afterwards.

I was unfortunately kicked out of it recently, but for the longest time I was in a WhatsApp group of Brazilians buying and selling US-based TikTok accounts, because the revenue per thousand views in the US is higher. So we've created an incentive mechanism for people in the Global South to trade sensationalist accounts in the Global North, especially the United States, and push fake news into it.

There's a poetic justice there — albeit a tragic one — that after exporting machines of mass disinformation to the world, the US is now reaping some of the negative consequences from the world. But yeah, I'm always in for a story about how the way we set up these platforms ends up incentivizing that behavior.

Rob: An interesting angle on that: there are areas of specialization across the globe. You mentioned Brazil; you can look at Pakistan, at Vietnam. Not to overgeneralize, but there are certain places where there are highly technically capable people, certain things have local traction, people talk to each other, and it keeps evolving.

The other thing I've noticed recently: we found some content out of Brazil that was bad, reported it to one of the platforms, and then the same actors came back and used the fact that they'd been taken down as the message — "They don't want you to know about this. They took us down, but we're back." That became part of the pitch.

We're at a super interesting point where things feel very unstable. Maybe the defenders will catch up with the attackers on their use of AI. But right now the evolution and capability cycling by bad actors is supremely high.

Alexios: And we haven't even seen the full rollout of ads on AI chatbots. That's coming, and it's going to have a major impact. It's here already, but not at the scale we've seen elsewhere. So this field will keep on giving.

Rob: One other set of journalists I've found very interesting is 404 Media. What sources should people interested in this kind of work be looking at — tools, resources, journalists — in addition to Indicator?

Alexios: Big fan — big stan, I should say — of 404 Media. One of my sadnesses about leaving New York is that I can no longer attend their annual parties as a paying member. Major shout-out to what they've built and continue to do.

There's a series of long-standing think tank and analysis teams doing work in this space: Graphika, DFRLab, the Institute for Strategic Dialogue. I pay quite a bit of attention when they put something out. There are individual reporters I track at places like Wired — Matt Burgess does a lot of work in the nudifier space that I monitor. Kolina Koltai at Bellingcat.

The good thing is that, especially on the worst harms, there's a strong sense of community. People share, even though notionally you're competitors — tips, or things they left on the table. There are really a lot of people trying to do good here.

What I will say is that I don't know about this fragmentation, and how much users will be willing and able to pay long-term for this work being done outside the platforms. We're a subscription-based publication, and we're doing just fine. But we've noticed that the training component drives quite a bit of conversion in a way the reporting sometimes doesn't. And we have crude ways of knowing that — it's coarse, the way you can determine it.

My question is whether we'll all be able to keep doing this work as funding dries up and readers may or may not understand the value of what's going on. Look at the fact-checking ecosystem: Meta pulled the plug on the program, and Google dropped many of its more grant-like funds. As problematic and limited as those programs were, they recognized that the content fact-checkers produced had economic value to the platforms.

That economic value is still there for X in Community Notes — fact-checkers are the second or third most cited source in Community Notes — but they get nothing for it. Similarly, I'd imagine, and I'm doing a long-term research project on this, that AI chatbot answers on disinformation are likely worse if you ablate and remove fact-checking websites from the training data than if you keep them in. But there's obviously no compensation for that.

So if anything, we're moving further away from an understanding that the information giants need to throw some crumbs to the fact-generating and verifying ecosystem, or it will starve.

Rob: I've seen Google, Meta, and other companies fund anti-scam efforts, and there are nonprofit initiatives in this area that have been surfaced and mentioned, though not necessarily visibly funded as far as I'm aware. There's a growing sense that we need to look more closely at the harms — and hopefully not just at the endpoint of the harms, since there are a lot of other pieces to this, as your work points out.

Hopefully we can get that better funded and get more attention on it. I see people who are very concerned about the underlying harms but aren't necessarily educating themselves, and the mainstream press doesn't always pick up on the more technically savvy folks who can comment on it directly.

Rob: What's next for Indicator? What might we expect in the next few months?

Alexios: We're always looking for ways not just to break stories but to help others investigate these platforms. One thing I'm excited about, which Claude Code has made possible, is converting all of our spreadsheets — all the stuff that undergirds our investigations — into a searchable library of bad actors.

We have thousands and thousands of accounts that spread scams, behaviors, and so on. Sometimes even I forget that we'd covered this platform or that one, or why we did it. We'd like to open this up and provide it as a service to our subscribers, because that historical context is valuable. And it's obviously much easier to structure this data semantically now than when you had to manually annotate and tag every single thing and maintain a taxonomy that may or may not withstand the test of time.

So — a bit wonky, and it might not get people's hearts racing, but that's what's on my mind.

Rob: I find it exciting. I think that'll help a lot of people do more investigative work and find patterns others have missed. So thank you for doing that.

It was really great chatting with you — this was super interesting. I'll have links to a bunch of your previous investigations and the website in the show notes. Any closing thoughts on what the platforms could or should be doing that they aren't doing yet?

Alexios: I'm just amazed at the short-sightedness of some of this anti-trust-and-safety backlash. So much of it was informed by US politics, and US politics is volatile. The steadier course some of the smaller platforms have been able to hold is probably much healthier — because otherwise they're going to have to rebuild entire institutions.

Trust and safety is here to stay. Most platforms should realize that.

Rob: Great. We'll leave it there. Thanks again, Alexios — always great to see you.

← All Won't Fix episodes