The Rules You Didn't Vote For

Transcript

There may be transcription errors: we apologize for those in advance.

Rob: Welcome back to Won't Fix. My guest today is Jonathan Bellack. For 30 years, he's been in leadership roles at internet companies — ones you've heard of, like DoubleClick and Google. More recently, he was one of the founding members of Harvard's Applied Social Media Lab. A year ago, he started writing a newsletter called Platformocracy, which we'll hear more about. His thesis is simple and uncomfortable: tech companies didn't set out to govern us, but they do now. Billions of people are subject to rules these companies make — rules they didn't vote for, enforced by systems they can't see, with no meaningful right of appeal. So we're going to get into that. Jonathan and I also worked together at Google; we were part of the same leadership team, and we'll talk about that too. I hope you'll enjoy the episode. And as always, give us your feedback. Thank you.

Rob: Jonathan, it's good to see you, sir. Has it really been that long since we chatted?

Jonathan: No, we've caught up. We've kept up pretty well in the three years since we took exile from the Plex.

Rob: That's right. So as I mentioned, you and I worked together at Google — we were part of the same leadership team. And one of the things we're going to talk about is Platformocracy, which is what you've been doing for the last… what is it, has it been a year already?

Jonathan: Almost a year. It'll be a year June 6th.

Rob: Oh, congrats. That's amazing. I'll have to buy it a cake.

Jonathan: Yeah, exactly.

Rob: So writing this newsletter, and hearing all the feedback you've surely gotten from readers — have you changed your mind about anything over the last year?

Jonathan: It's a great question. It really is the classic case of any overconfident, middle-aged man who wades into a topic he knows a little bit about being humbled by the amount of complexity out there — and the number of people who've already thought deeply about these areas, not just in the technology world but in world-historic terms. I think I've spent as much time reading old political science books and manifestos, trying to educate myself on theories of democracy, as I have learning about the latest identity technology, or how we're going to use zero-knowledge proofs, or what the latest legislation is going to be.

Rob: I can imagine some of this was quite different from the kinds of conversations you may have had at Google. You talk a lot about participatory governance. Have you been involved in discussions about that, at Google or elsewhere? We could also talk about the Applied Social Media Lab at Harvard, which would be interesting. But have you had any meaningful, productive conversations about this inside the machine versus outside the machine? Or is it something people don't really want to talk about?

Jonathan: I think it's almost a category problem — whether it's even a relevant discussion in a commercial sense. And I don't want to demonize any individual technology company, but the dominant metaphor for thinking about these community platforms — whether it's Meta's platforms like Facebook and Instagram, or YouTube, or TikTok, or Snapchat, or even ones with multiple smaller communities inside them like Discord or Reddit — is very much that they are a business, and they're entitled to run their business however they see fit. And your recourse, if you don't like it, is to become a customer of another business. Which is, of course, how capitalism works, and there's nothing wrong with that. I have an MBA. I'm a card-carrying, proud capitalist. I'm not saying we need to turn the means of production over to whatever, whatever, whatever.

But the notion of what these organizations are is challenging to that model. You'll hear tech executives say, "Oh, it's just like running a bar — we have the right to refuse service." Or, "We're a store; as long as we're not violating anti-discrimination laws, we're allowed to decide who we do business with." That misses the fact that these are also communities of humans — many of whom already had real-world relationships and chose to use these platforms to maintain them, or who built real-world relationships using the technology of these platforms. So when somebody says, "Oh, it's the Facebook community" — it's not the Facebook community. It's the community of people I went to college with who are choosing to use a Facebook group to stay in touch. We don't belong to Facebook.

And that's the hard thing when you're trying to talk to people inside these companies. There hasn't really been a readiness to think of the community as having rights, standing, or power independent of the sole choice to stop using the platform and go somewhere else.

Rob: Yeah, exactly — you're locked into being there, or at least that's where your community is. One of the things you talk about is that people, and companies by extension, authentically want to do the right thing. But is this a case of wanting to do the right thing and then essentially ending up as an unelected government? How do you reconcile those pieces — the individuals, the companies, and the governance that comes out of it?

Jonathan: There are two ways to look at this, and I don't want to give you a 20-minute answer — so let me try. It's that line from The Princess Bride: "Let me explain. No, there's too much. Let me sum up."

The best-case way to look at it: you have communities, all the way back to the beginning of digital communities. You can go back to the virtual community book — I think it's Douglas Rushkoff who wrote it — an amazing book about the WELL, Minitel in France, and early electronic communication systems in Japan, all around the world. The reality is that any group of people is going to have conflict, is going to have people who don't want to live by the group's rules, who become disruptive, who fall out with each other, who show up trying to hurt the community. That's inevitable, and it makes sense that you need something there. It's hard to say you'll have an anarcho-collective where everyone's off on their own, there are no rules, and everything sorts itself out as long as we all talk to each other enough. That's an unsatisfying solution in a lot of ways. So you need someone to say: what are the rules? How do they work? How do we help people work together? You had community managers back in the days of AOL chat groups and Usenet and so on.

When these organizations started scaling these communities to unbelievable levels — from hundreds to thousands to millions to tens of millions to billions of people — you get a reactive sense of, "Oh my goodness, having individual community managers isn't getting this done. We need to do more." "Oh, wait, we're now so big it's worth criminals exploiting us to take advantage of people, or steal money, or steal our data." You have horrible people discover this is a really efficient way to share horrible images and horrible content and plan horrible things. And people react: "Oh, crud, what do we do? How do we scale this up?"

Some of my favorite people in the world are trust and safety people — whether you call it integrity or trust and safety. These are incredibly dedicated people giving their lives over to reducing harms, keeping people safe, making things better. But because of that corporate framing, they skipped the whole step of who should be doing this? How do we make these decisions about what's right and wrong? There wasn't a step of thinking through what this implies about the community and society we're building.

And then you get the intersection where there's now an enormous amount of money in it. You have the lawyers and the businesspeople and the policy/lobbying/PR people getting involved about what's good for the business — which isn't necessarily what's good for the community of humans who don't have a voice in where the decisions go. In aggregate, you end up with what feels, to the people in these jobs, like the best they can do — but maybe isn't the best the communities using these platforms should get.

Rob: One thing I think is sometimes missing, in my opinion, is pragmatism about who's paying the bills — what's funding the exercise. Within that framework you can do a lot. But I'd sometimes hear from people at these companies who'd say, "Some folks see it as their job to protect the world from the company," as opposed to… I think it's important to have some of that on a team working on this stuff. But you need bits and pieces of a variety of viewpoints — just as in any government or democracy, you need different viewpoints and to reconcile them with policy solutions and with how you do things. So this is a super fruitful, interesting set of topics. How did you land on wanting to spend your time on this? I love your writing — sometimes I feel like to fully appreciate it I need to go buy a book you recommend and read it. It's very well written and researched. What was the process, going from what you were doing at Google through the last couple of years, and deciding to focus on this?

Jonathan: Great question. I left Google at the beginning of 2023 — I actually gave notice in November of 2022 and ended up leaving in the first big round of layoffs, through mutual agreement with my manager at the time, which gave me a chance to think about what I might want to do and take a deep breath. I joke that after 30 years in technology, if I'd been a public servant I probably would have retired with a pension at that point.

I had an opportunity to help the Berkman Klein Center for Internet & Society at Harvard build a new lab called the Applied Social Media Lab, thanks to a great gift from a group called Project Liberty. I spent about a year doing that, and it was really rewarding to be back to trying to make something happen, as opposed to poking the dials of a gigantic organization. I did it for a year and then realized I'd hired this amazing woman named Meg Marco — who'd been at ProPublica and Wired and, I think, the Wall Street Journal, a bunch of places — and she could run the place as well as, or better than, I could. I didn't need to sit there and take up space, so I handed it off to her.

The reason I mention this: one of the best things I got out of academia that you don't have in corporate America is a community of people who are really interested in your ideas and want to hear new ones, learn from you, learn from each other, explore things. In business, you'd criticize a meeting by saying, "Oh, we just picked up the problem, looked at it from different angles, and put it back down. Why are we here? We're not getting anything done." In academia, large swaths of it are all about picking up the problem and figuring out the right words to talk about it.

I gave a talk while I was there that started crystallizing what I felt had gone wrong with the world of the internet — which I've loved deeply since I discovered it in college — and I was pleasantly surprised that people reacted with, "Hey, you might be onto something." One very prominent professor — I don't want to name-drop — said, "I loved your speech; I agree with every word." And I'm a sucker for that kind of ego-feeding, so I thought, okay, let's try it and see where this goes.

I'd also say what's at the heart of what I'm trying to do: I'm not a big believer in tearing everything down as a way to build the future. It's very easy to talk about how terrible everything is, point out problems, laugh at people, and say if we just turned everything off we'd be happier. There are lots of thinkers who've built careers on how terrible everything is. That's fun and lucrative — you can do it for profit. What's harder, and more interesting to me, is to say: given the reality of where we are and the changes that have happened and will keep happening, what are some ideas that might help us build forward? That's been the great thing about the newsletter — the research, the people I've met — starting to find people who are a little beyond just talking about how terrible things are, and are trying to say, "Here's something that might make it better."

Rob: I love that. And it leads me to something you wrote in a recent post — you hinted at it earlier — about decomposing social media into different things: parasocial media, social media, and social networking, and what percentage of activity is which. Why is that interesting? Tell me more about that distinction.

Jonathan: These are ideas from two really great thinkers you should follow. There's Ben Werdmuller — I forget if he's the CTO, but he's a leader at ProPublica, the great investigative-journalism outlet. He's built social media platforms as a founder and CEO, he's been an investor, and he writes regularly about these topics. He did a keynote at a FediForum back in October. FediForum is a gathering of people involved in the Fediverse — Mastodon, ActivityPub, everything around it — that happens a few times a year online to share ideas.

In that talk, Ben introduced the idea that social media is the consumption experience — sitting there flipping through TikTok, Instagram Reels, YouTube Shorts, doom-scrolling or hope-scrolling or whatever-scrolling you enjoy. That's a form of media, even though there's user-generated content behind it. Whereas social networking is actually building community. In his keynote he gave a longer example: say you're trying to resist an unjust government action. You start by getting together with a very small number of people you can absolutely trust, and you need secure encryption for that. Then maybe you start connecting with other groups that have formed to resist. Maybe at a certain point you decide to share something out into social media — but you're building and managing a community. You want privacy, methods of trust, some control. You don't want to feel like the whole thing could get turned off by an enormous corporation thousands of miles away.

And then danah boyd — she's a legend. Her book It's Complicated, about the social lives of teens, is probably the best antidote I know to the alarmism that phones are destroying teenagers. As the title says, it's more complicated. She put out a piece a week or two ago that I thought was great, where she said social media is now parasocial media — pointing out that a lot of what you experience on the media side is the simulation of community, but you're really just interacting with a creator at arm's length. Watching a bunch of Mr. Beast videos doesn't make you Mr. Beast's best pal. There are people I love — Hank Green, Patton Oswalt, a comics creator named Gail Simone — all these great people on Threads and Bluesky right now. I love what they write, I laugh, I think it's clever. None of them know who I am. I don't have a real relationship with them. It's all parasocial.

What I liked about putting these two together goes back to the notion that these communities — these corporate entities — are something new and different. They're not power utilities. They're not just the phone company. They're not just regular corporations. So how do you talk about what's underneath them? Drawing these distinctions — what are the different use cases, and what should governments have in terms of rules — becomes a more productive way to talk about it. When you talk about age verification and age restrictions for social media, you have to be clear about what problem you're trying to solve. Is the issue algorithmic feeds with an addictive character? Is it content inappropriate for young children being displayed to them? Is it the risk of adults communicating with and harming children? Is it children communicating with each other in harmful ways? Each is different and might have a different appropriate solution — not just, "Let's turn the thing off until you turn 16."

Rob: I was going to ask about age verification, because you've spent a lot of time writing about it — and you have views some people agree with and some don't, which are the best kind of views. As a lead-in: I met some people in Australia, some of whom worked in government and some in technology, and one of them was talking to me recently about what they've learned from the social media age ban. The TL;DR was that they hadn't heard much negative about it — and that was part of their worry. It seems it was very quickly circumvented by the kids who cared to keep going. As you know, there are lots of ways to get around technological restrictions. So I'd love to hear more about your views on age verification, because it's a deep and meaningful topic for all of us who grew up on the internet, where nobody knows you're [blank].

Jonathan: It's a big topic, and I don't want to speak authoritatively as though I know the right answer. My biggest concern about age verification is the notion that we'd require every adult in a given jurisdiction to submit identity information — however securely you say you'll do it — in order to protect a different class of people. People look at it like, "If you want to buy alcohol or tobacco, you show ID. Why is this any different?" The difference is the pervasiveness of these platforms. I was digging around trying to find statistics on what percentage of people buy and drink alcohol, or buy and use tobacco — it's not as high as you might think, relative to the number of people using social platforms to communicate. So it's one of those things that sounds good, but then you realize you're putting enormous chunks — if not the entire internet — under an identity regime. And no matter how much you say you're doing it safely with zero-knowledge proofs, you're still saying you have to prove something about yourself to use even part of the internet — a substantial part, because again, we use it for communication. That's where I start. I'll pause there before I get into other issues.

Rob: That's very well said. One thing a lot of folks looking at this don't reckon with is that the things that seem like they should just work turn out to be hard technical problems. I worked on age verification at Facebook when we were doing political ads, and we had to explain to people that even in the countries with the best coverage, I can't be even 90% accurate on age. And that's mainly for adults — once you're talking about kids it gets very tricky. Over the last couple of years I've talked to a lot of people implementing this technology at other companies, and it's, "Yeah, there's this vendor that does this thing, and they claim it's this, but it's actually that." It turns out there's a lot that's hard — and if you try to explain that to a regulator, a litigator, or a journalist, they say, "Well, of course it's in your interest to say it's hard." But I do think age verification is uniquely — maybe not uniquely, but certainly quite — difficult.

Jonathan: Yeah. And the question of the appropriate role of governments, as opposed to the communities on these platforms, is interesting — because a lot of the people in government talking about this don't use these platforms. They're not familiar with them, not deeply embedded in them. So you're being governed by representatives of the people in one way, but the actual interests of the communities on these platforms aren't being represented.

Historically, we're really still in a fairly early stage of human societies changing in a deep and profound way due to the internet and internet technologies — like the phone, and, God help us, AI. I harken back to things like the advent of printing in Western Europe. There's a great book, The Printing Revolution in Early Modern Europe by Elizabeth Eisenstein. There's a long version I don't think anybody reads, and a shorter one — the version I read — which is surprisingly readable for 500-, 600-year-old history. She talks about how much the whole notion of society and humanity shifted once you had printing. It changed the role of the Christian church in Western Europe. Who determined what truth was, once you had multiple versions of the Bible going around? And then scientific texts people could learn from and build on.

I think a lot of that is happening with the internet. You and I are effectively immigrants to the internet — we can remember a time in our lives when we didn't have it. And you understand this better than I do, because you've actually moved from one country to another; I'm an American, so I don't have that. But think of the immigrant experience: you adapt, you learn, but you always have something else. Our children are the children of immigrants — that next generation still experiences a bit of a dual culture, the country where they were born and are citizens, plus the culture of the parents who came from somewhere else. It's really the generation beyond that — our grandchildren, our great-grandchildren, raised by someone who can't remember a time before the iPhone — whose way of looking at these problems will be very different. Hopefully either better informed, or misinformed in more interesting ways, so we can talk about solutions rather than trying to apply that pre-internet framing forward and missing all these technology challenges you're describing.

Rob: I think about this a lot — how different generations think about things and react to technologies. You mentioned AI; I think of how the kids these days use AI more than they seem to like AI. There's a lot there. But you talked about the printing press and the internet — this idea that we now have humans using tools to create things that aren't real, that simulate reality, deep fakes, AI slop. I think it changes the dynamics of some social relationships and our relationship to content — how we consume it in these networks. I'm curious how much you've thought about that, and what it changes in the parasocial-versus-social-versus-social-networking framework.

Jonathan: I'm going to take this question of AI slop, and of things that aren't real, in maybe an unusual direction — the work I did in my last role at Google. I was leading product management for a central counter-abuse technology team, dealing with compromised devices, compromised accounts, child safety, spam in products like Gmail — all kinds of harms, exploits, and so forth. I'd already seen some of this back in a product-management role in ads, where I saw a lot around ad fraud, but I really got immersed in the last couple of years.

Something I came away with: an enormous percentage of the harms we associate with the internet come from these organizations making it trivially easy to get an anonymous account that can do pretty much anything. Look at the spam out there — it comes from these things. The pig-butchering scams I know you're very concerned with these days. A lot of what people mean by misinformation and disinformation — state-sponsored disinformation is real, and it comes from governments and non-governmental entities being able to get hold of an account and pretend to be someone they're not. AI slop, in a lot of ways, is just a massive turbocharging of that gaping hole in everybody's security posture.

To call out one company: there were leaks that people inside Meta estimated something like 10% of Meta's ad revenue was coming from scams and fraud. And there were other leaks about the company choosing not to be more aggressive about closing some of these loopholes. I obviously never worked there, I can't vouch that any of these leaks are accurate — all you lawyers out there, please don't sue me, I'm just repeating what I've seen covered. So, to almost argue with myself on the concern about age verification and identity: we are going to need a conversation about how we can have more confidence that we're interacting with actual human beings when someone or something claims to be one. Actual human beings can still be terrible to each other — it won't create a utopia. But we've created a world… you were joking about the old New Yorker cartoon: "On the internet, nobody knows you're a dog." Everybody thought it was funny. And ironically, that's kind of at the heart of what's tearing the internet apart. Maybe we do need to know if you're a dog.

Rob: So it's the old "freedom of speech, but not freedom of reach." Is it essentially that argument, or a version of it? You probably hate that one.

Jonathan: I don't like that one, because I can't see past — again — who's making the rules. "Freedom of speech, not freedom of reach" is generally an excuse for saying, "We, the company, get to decide how our algorithm works, and you just have to put up with it." The problem is we don't have a voice in these algorithms. We don't even have the transparency to understand what they're doing.

I'll pick on my old boss to be funny. Neil Mohan was my boss in ads for 10 years, and he's a fantastic person. I have absolutely no idea whether the YouTube recommendation algorithm has a "Mohan circuit" in it — where if you want to say "Neil Mohan is a blankety-bleep-bleep," that gets downranked. I don't think it does; knowing Neil, he's a very honorable person. But that comes down to the fact that I know this man personally and can judge his trustworthiness. Someone who's never met Neil Mohan has no way to have confidence that YouTube is doing this accurately or honestly, or being a fair broker in what they see. That's the fundamental problem. It's not "Is Neil good? Is Mark bad? Is TikTok controlled by a foreign government?" The issue is we don't have any way to know or make any of those decisions.

If we knew any of that, we could start having a discussion about who's allowed to do what, and what we support and don't. We can debate — maybe consumption should be very anonymous. You should be able to watch what you want on Netflix or YouTube without the government, or even the company, needing to know who you are. Maybe if you want to publish to a small number of people, that's fine. But I don't know that you should be able to publish something that could reach billions of people virally without some level of… something. Even saying that, I know civil-liberties people will be howling. And I want to be clear — I'm not proposing a glib "we just make everybody log in before they post big things and it'll be fine." That could create all kinds of problematic censorship issues too. But that's the kind of debate we have to start having, rather than just turning it over to the platforms and yelling at them when we think they got it wrong.

Rob: That makes sense. In a sense, the ranking algorithm — folks like you and I who've worked in these companies have always known ranking algorithms are very important. The public has come to realize that, and maybe now they're far more important. If you have the volume of content and the provenance of content — volume has gone up, provenance has gotten sketchier — things can still go viral, but we haven't caught up in other areas, whether identity or otherwise. So the ranking algorithms become even more important. We also have liability regimes that protect the companies from being sued or held accountable for what goes viral. So does it feel like there's a mismatch now in how we handle or think about algorithms as a society? Many of these algorithms are based on millions of rows in a database — no one could really explain them without building another algorithm to explain the model. It's technically tricky, but I wouldn't say it feels powerless. What's your take on the algorithmic side?

Jonathan: Lawrence Lessig — the famous Harvard professor and thinker — coined the concept that "code is law": when you're writing code, or having an algorithm learn to make decisions enabled by code, you're setting up the laws of that community, what works and what doesn't. Going back to the earlier question of how we ended up in these "governments" — just as the trust and safety folks are great people doing the right thing with the access and power they have, but in aggregate it's problematic, I think it's the same with the people building the algorithms. They're trying to achieve their objective function: don't show people horrible things, show them things that are good and that they're interested in, and — probably at some point — support a business goal around driving engagement or increasing usage or making more from ads. But in aggregate, to your point, we've created something large, powerful, and important that we haven't really been able to have a discussion about, and can't really choose among or opt out of.

Rob: When you think about AI helping with content moderation and trust and safety — I'm trying to find something optimistic in every conversation, and not all of it is optimistic. I don't think you're not optimistic. But how do you see AI's role in helping keep platforms safer, or maybe taking decisions out of the hands of a small group of unelected governance folks? Is there some optimism, some things to point to that give you hope?

Jonathan: I'm very hopeful we'll keep seeing advances in personal AI — on-device models, technologies to help people manage their own lives in ways that feel more natural. Go back to child safety. Part of what makes it feel so hard for parents right now — and I'm very sympathetic — is that every platform has slightly different child-safety tools, in different parts of the app. How do you set up your family? How many separate parent accounts do you need to pair with your kid accounts, to say what they're allowed to do, what they're not, what age they are, how these things work together? The classifications are challenging. I think Roblox has one classification for "frequent unrealistic blood" that's different from "infrequent realistic blood." What does that even mean? How does a parent decide what's okay for an eight-year-old versus a twelve-year-old? It's messy and overwhelming, and I understand why it feels infeasible to keep up with.

In the real world, if you've got an eight-year-old who says, "I want to ride my bike to my friend's house," you can think through all the safety and planning considerations in your own mind, in natural language. Which kid? How far away is it? Does my kid reliably wear a helmet? Are there busy streets? Is it a small suburban neighborhood? Does it look like rain? What time is it? When's dinner? And you can come back and say, "No, sorry, it's too late," or, "You can go, but you're not allowed to go to the candy store together, because last time there was a problem — and be back by five for dinner." You don't have to rely on the bike company to ship an update — "We've added time-zone support to your Schwinn, now everything's great." "Peugeot, now with a GPS chip." You don't rely on the provider.

So this is where AI gets interesting. Right now everybody's using it to organize their calendar of professional events and optimize their time. Why can't you apply that to your kids? "Here's the deal: I don't want them to see any violent content on YouTube, and I don't want them seeing people insulting each other while playing video games — that's just not what I want. But they love professional soccer. They can watch any professional soccer content, even a gruesome on-field injury, even people talking smack in the stands. That should be possible." YouTube has done more than probably any platform — there's YouTube Kids, there's YouTube itself, each with multiple levels of control — but I can't do what I just said. I can't just tell YouTube what I want my kid to see and not see. I'm very optimistic that's the sort of thing AI could enable. I could take my kid's iPad and say, "Make this unavailable except for playing podcasts after bedtime on school nights." And if there's a late opening tomorrow, I should be able to pick it up and say, "Hey, iPad — this is the voice you've already verified locally, this is Dad — add an extra hour today because it's a late school opening." I think AI could make it way easier to be a parent online.

Rob: I think that's true. I've been experimenting with getting Claude Cowork to summarize the many, many emails from the kids' schools — and that's going better than I thought so far. We'll see how it scales. This is May, the time of year when there's so much stuff; everyone's trying to finish up the school year.

Jonathan: So it'll come back and say, "I realized you were getting too many notifications about your children's things, so I decided the best thing to do was auto-delete half of them so you wouldn't feel so overwhelmed."

Rob: Yeah, exactly. So one of the ideas I had about 10 years ago — I talked to someone about building an Android box you'd plug into your home TV setup. Because I was thinking: how do you let kids watch sports, but not see violent ads for violent video games? It wasn't really ad-blocking, more like blocking content that obviously — for a lot of reasons — doesn't make sense. No one's going to start a real business off that. But there's definitely something here about personal AIs. These aren't novel ideas — science-fiction writers like Neal Stephenson have written about this for years. But I think we're getting to a point where you could actually implement some of it — even with a bunch of tokens and a weekend, you could probably build an interesting proof of concept.

Jonathan: I think you can get there, and this is where you need hooks. There's a great trust and safety leader out there, Alice Hunsberger — her latest piece this week is about how we need to standardize parental controls, which I think is an interesting idea. And this goes back to age verification. Instead of spending all this time trying to figure out exactly who everybody is, what if we turned it around and said: "Hey, company, hey, platform — if you want to serve children, you take on an enhanced set of obligations, because you're asking to do business, make money, and serve a population that has special status in our society for a lot of good reasons."

Put the burden on them. Like some of the recent cases Meta and YouTube have lost over harms to children: if you want to serve kids, you have to provide highly usable parental controls that can be automated. You have to take a different level of responsibility for moderating your content. And, again, kudos to YouTube — the first version of YouTube Kids had a lot of problems, and they went back and changed their approach to get more of the terrible things out of there. I'm not sure where it is right now. But then: if you don't want to incur the cost and difficulty of creating something with an appropriate level of risk for kids, then you need to make clear your product is not for kids, and don't let kids' devices use it. People will say, "That just loops back to how you know if they're a kid." And it's a bit of — okay, then say "I'm only for adults," or maybe we have it on the device: if the account on this device has been labeled a kid's account — recognizing not everyone has separate devices — you need to do something different.

Rob: I like that. It's kind of like — if you're a developer who wants to build high-end condos, you also have to build some affordable housing, right?

Jonathan: That's one variant of it — perhaps not exactly what you were saying. I think it's more like: if you want to open a restaurant, you have to meet the health-inspection standards. If you want to open a liquor store, you're liable for making sure you're not selling to 15-year-olds. And if a store owner says, "Well, I can't do that" — then you don't get to do it. I took that as a given.

Rob: Yeah.

Jonathan: It's funny — you go back to these questions of the commercial imperatives and the practicalities. What I don't accept is that these platforms seem to be saying, "Make our products as safe as possible — as long as it doesn't impact our spectacularly large profits." Whereas in most important industries in the world — finance, building an automobile, opening a restaurant — the societal standard is usually, "We want you to be as profitable as possible, as long as you're meeting a baseline of safety." We've somehow gotten away from that online. These companies aren't barely scratching by, worried about giant layoffs just to keep the lights on. We're talking about some of the most profitable entities in the history of humanity, with multi-billion-dollar war chests. They could significantly expand their investment in any of these areas — public participation, safety, transparency, more options — and still be phenomenally profitable. And given that AI is making software development more productive, they could plow that money back in. I don't want to sound too glib, but we've lost the connection between profit and safety in this industry.

Rob: A hundred percent. You and I have had this conversation a couple of times about the Insurance Institute for Highway Safety and the objective standards for automobile safety. They started doing crash testing. They had a financial interest in safer vehicles, but they weren't the people making the vehicles. I think you need something like that to set objective guidelines — and then, sure, be as profitable as you can be, as long as you're meeting the safety, transparency, or whatever guidelines. But we're stuck in a chicken-and-egg situation until people realize we need to solve one side of it. The baseline question needs to be addressed.

Jonathan: And the other thing I'm very optimistic about — whether AI-enabled or in general — is that we could do a lot better on many of these things while still letting billionaires exist and thrive and become unbelievably rich. I don't think it's an either/or. It's a question of agreeing on how we handle all these externalities that companies currently aren't paying for — the harms — while still giving them plenty of room to exist and be phenomenally profitable. Coming up with standards around providing electrification to rural areas didn't come at the cost of utility companies all going out of business. You could still make a lot of money doing a lot of things in the world.

Rob: I agree. Well, this week is Google I/O, if I remember correctly — we had some fun discussions about that, and some fun work to do on it back when we were at Google. If you could put one question to Sundar, Zuck, or Elon on stage, and have them actually answer honestly — what would it be?

Jonathan: The second part of your question is doing a lot of work there. The couple of times I had the opportunity to ask a pointed question of a senior executive, the main thing I learned is that they were far better at dodging hard questions than I was at asking them. I'd almost resist that framing. What I'd rather do is sit down with one of those people behind closed doors and talk about what's actually going on — try to negotiate around changing the incentives on some of these things and find a new, healthier way forward. Where they can trust that this isn't an uninformed elected official or bureaucrat trying to mess up the good thing they've got going, but also saying, "Hey, we're building something that affects the future of our societies, our countries, our civilizations. Why be satisfied here? Why not try to do better?"

I enjoy how Nilay Patel frames it on his podcast — it's execs versus media training, him versus the exec's media training — because, again, they're so good at answering things that way. The part I wish we could have, though it may not be realistic, is actual real discussion about this stuff in public, or with folks who get it. So much of this — especially, maybe, in our government — is performative question-asking in front of the nation. I'd prefer we figure out some way to make progress on these questions and get people in the room who, a priori, we might not know should be in the room. To your point about participatory governance, there's a lot to unpack in who gets to participate in that discussion.

It's funny — the latest thing I've been reading is about different models of justice systems, which I wasn't super familiar with. I grew up in the US, which is heavily influenced by the UK common-law tradition. Continental Europe has a different approach that comes literally out of ancient Rome — the civil-law tradition. I'm now in the section on the history of Chinese law, which I didn't know much about. One of the distinctions in law is between substantive law and procedural law. Substantive law is "thou shalt not kill" — if you steal my stuff, you've done something wrong; you can't break a contract. Procedural law is how the laws are made: jury trials, what the government has to do before it seizes your property.

I think part of why these conversations with companies aren't very productive is that there's a lot of focus on the substantive: "Kids are being harmed — you have to stop that." "There's fraud — don't let there be fraud." It's all about the outcomes we want. There's not nearly as much time spent on the procedural part: What can these companies do? Should there be a public notice period before they change their policies? Should you be entitled to due process before your account is terminated? I think the Digital Services Act in Europe has started to nod in this direction — some transparency requirements, and a notion of being able to appeal certain decisions, though it's still very tilted in the company's favor. But at least it cracks it open. Maybe we could agree on some procedural aspects that still give the company a fair chance to be involved and say things — but that says, "It can't be you making this decision behind closed doors and then telling everybody ex post facto why you think you got it right." Critics might not get their way, but at least there'd be more of a voice.

Rob: I think that's an optimistic note to end on.

Jonathan: I believe there's something there we could figure out.

Rob: I do agree with you. How should people find you online? Where should they go?

Jonathan: Absolutely. You can subscribe to my newsletter, Platformocracy — a free, mostly weekly newsletter that advocates for more democracy online — at www.platformocracy.com. That's p-l-a-t-f-o-r-m-o-c-r-a-c-y dot com. Hope to God I spelled that right.

Rob: Yes — we'll put it in the show notes as well, so people will get it. That'll be great.

Jonathan: And you're welcome to email me directly at jonathan@platformocracy.com. Happy to hear from anybody who's listened, and I'd love to hear your thoughts and see if I can help.

Rob: That's awesome, Jonathan. Thank you so much. It was really great to catch up with you again — I always learn something when we chat.

Jonathan: Thanks, Rob.