# AI Whistleblower: We Are Being Gaslit By AI Companies, They’re Hiding The Truth! - Karen Hao

- Channel: [The Diary Of A CEO](https://www.youtube.com/channel/UCGq-a57w-aPwyi3pW7XLiHw)
- Published: 2026-04-01
- Source video: [Watch on YouTube](https://youtube.com/watch?v=Cn8HBj8QAbk)
- Read on ReadTube: https://www.read.tube/p/videos/Cn8HBj8QAbk
- Language: English

## Summary

**AI Empires Hide Costs**

Karen Hao argues that today’s AI industry functions like an empire: it extracts data, labor, land, and public attention while concentrating power in a few companies. She says the biggest firms use shifting claims about AGI, existential risk, and future abundance to win money, dodge regulation, and shape the public narrative. Hao also says the same technologies could be built in less harmful ways, but the current model is driving layoffs, environmental damage, and tighter control over knowledge and research.

Karen Hao says the AI boom is not just a technology story but a power story. After moving from mechanical engineering into tech journalism, she spent years reporting on OpenAI and the wider AI ecosystem, interviewing more than 250 people and tracing how the industry grew around vague promises about artificial general intelligence. In her view, the central problem is not that AI has no value, but that the way it is being built right now imposes serious harm while concentrating decision-making in the hands of a few leaders and companies.

She argues that AI companies resemble empires in several ways. They claim data and intellectual property that are not theirs, rely on large amounts of global labor to train and deploy systems, buy influence over research and policy, and present themselves as the only responsible actors able to manage a powerful technology. Hao says figures like Sam Altman and Elon Musk helped popularize a myth of both catastrophe and salvation: AI could destroy civilization, but only their companies can safely build it. She says this rhetoric is persuasive, internally strategic, and often used to justify anti-democratic control.

A major thread of the conversation focuses on OpenAI’s culture and leadership turmoil. Hao describes how Altman’s style polarized colleagues, how senior figures such as Ilya Sutskever and Mira Murati came to believe he was creating chaos and inconsistency, and how the board’s brief firing of Altman reflected deeper concerns about governance, not just personality. She also argues that major AI firms shape research agendas, censor inconvenient findings, and use access and intimidation to manage journalists and critics.

Hao rejects the idea that brute-force scaling alone will inevitably produce broadly intelligent machines that replace most human work. She says today’s models are narrow, jagged, and shaped by commercial priorities, with companies choosing which abilities to improve based on what is most profitable. She sees the real social effect as uneven: some executives and knowledge workers gain leverage, while many others are pushed into worse jobs such as data annotation or out of work entirely. She also warns that data centers and supercomputing facilities can damage local communities through power use, water demand, and pollution.

- The AI industry is, in her view, an empire built on extraction and control.
- AGI is treated as a flexible slogan rather than a clear scientific destination.
- OpenAI’s internal politics reveal a larger crisis of governance and accountability.
- Job loss is already happening, but the deeper issue is an unequal restructuring of labor.
- Hao’s answer is not to stop AI entirely, but to break up concentrated power and build smaller, more useful, less harmful systems.

## Article

### AI industry harms and empire myths

So much of what is happening today in the AI industry is extremely inhumane.

> **But this is me playing devil’s advocate.** And logically, it could be the case that the civilization that accelerate their research with AI is going to be the superior civilization.

> **No, it’s not.** This is a prediction that you’re making, right?

> **Making Zuckerberg’s making.**

> And do you know what the common feature of all of them is? They profit enormously off of this myth. You know, I have all these internal documents showing that they’re purposely trying to create that feeling within the public so that they can extract and exploit and extract and exploit.

So, what do we do about it?

> **We need to break up the empires of AI.**

I’ve been covering the tech industry for over 8 years, interviewed over 250 people, including former or current OpenAI employees and executives. And I can tell you that there are many parallels between the empires of AI and the empires of old.

First, they claimed the intellectual property of artists, writers, and creators in the pursuit of training these models. Second, they exploit an extraordinary amount of labor, which breaks the career ladder because someone gets laid off and then they work to train the models on the very job that they were just laid off in, which will then perpetuate more layoffs if that model then develops that skill. And when they talk about that there’s going to be some new jobs created that we can’t even imagine, a lot of the jobs that are created are way worse than the jobs that were there.

And then there’s the environmental and public health crisis that these companies have created, and how they’re able to also spend hundreds of millions to try and kill every possible piece of legislation that gets in their way, and will censor researchers that are inconvenient to the empire’s agenda.

What I’m saying is not that these technologies don’t have utility. It’s that the production of these technologies right now is exacting a lot of harm on people. But we have research that shows that the very same capabilities could be developed in a different way that doesn’t have all of these unintended consequences.

So let’s talk about all of that.

This is super interesting to me. My team gave me this report to show me how many of you that watch this show subscribe. And some of you have told us, according to this, that you are unsubscribed from the channel randomly. So a favor to ask all of you: please could you check right now if you’ve hit the subscribe button if you are a regular viewer of the show and you like what we do here.

We’re approaching quite a significant landmark on this show in terms of a subscriber number. So if there was one simple free thing that you could do to help us, my team, everyone here, to keep this show free, to keep it improving year over year...

### Journalism path into Silicon Valley

I took a strange route into journalism. I studied mechanical engineering at MIT, and when I graduated I moved to San Francisco and joined a tech startup. I became part of Silicon Valley and, in a sense, I basically received an education in what Silicon Valley is about.

A few months into joining a very mission-driven startup that was focused on building technologies that would help facilitate the fight against climate change, the board fired the CEO because the company was not profitable. In hindsight, that was a very pivotal moment for me, because I thought: if this hub is ultimately geared toward building profitable technologies, and many of the problems in the world that I think need solved are not profitable problems, like climate change, then what are we actually doing here? How did we get to a point where innovation is not actually necessarily working in the public benefit, and sometimes even undermining the public benefit in pursuit of profit?

In that moment, I had a bit of a crisis. I thought, well, I just spent four years trying to set myself up for this career that I now do not think I am cut out for. So I thought I might as well just try something totally different. I have always liked writing, and that is how, after two years, I landed a role at *MIT Technology Review* covering AI full-time. That gave me a space to explore all of these questions: who gets to decide what technologies we build, how money and ideology also drive the production of those technologies, and how do we ultimately make sure that we actually reimagine the innovation ecosystem to work for a broad base of people all around the world.

That is kind of how I then set off on this journey of ultimately writing a book. I did not realize that I was working toward writing a book, but starting in 2018, when I took that job, was essentially the moment in which I began researching the story that I document in it.

A very timely time to start working in artificial intelligence. For anyone who does not know, this was before the OpenAI ChatGPT launch moment that shook the world. But in writing this book, you interviewed a lot of people and went to a lot of places. Can you give me a flavor of how many people you interviewed, where it took you around the world, and so on?

I interviewed over 250 people. So, over 300 interviews. Over 90 of those people were former or current OpenAI employees and executives. The book covers the inside story of OpenAI’s first decade and how it ultimately got to where it is today.

But I did not want to write a corporate book. I felt very strongly that in order to help people understand the impact of the AI industry, we would also have to travel well beyond Silicon Valley. These companies tell us that AI is going to benefit everyone, and that is their mission. But you really start to see that rhetoric break down when you go to the places that look nothing like Silicon Valley, that speak nothing like Silicon Valley, and that have a history and culture that are fundamentally different as well. That is where you start to really understand the true reality of how this industry is unfolding around us.

### How AI and AGI are framed

In mind, I am going to ask you where this journey begins, and where we should be starting if we are talking about the subjects of *Empire of AI*, AI generally, artificial intelligence, and also one thing I am really keen to do in this conversation, which is that I often see in conversations, left out, is: let us assume that our viewers know nothing about AI.

> Yeah. So they do not know what scaling laws are or GPUs or compute or whatever, and let us try and keep this as simple as we possibly can in terms of language, or explain all the complicated language so that we can bring as many people with us as we possibly can.

> Yes.

Where should we start?

I think we should start with when AI started as a field. This was back in 1956, and there were a group of scientists who gathered at Dartmouth University to start a new discipline, a scientific discipline, to try and chase an ambition. Specifically, an assistant professor at Dartmouth University, John McCarthy, decided to name this discipline *artificial intelligence*.

This was not the first name that he tried. The previous year he tried to name it *Automata Studies*. And the reason why some of his colleagues were concerned about this name was because it pegged the idea of this discipline to recreating human intelligence. And back then, as is true today, we have no scientific consensus around what human intelligence is. There is no definition from psychology, biology, or neurology. And in fact, every attempt in history to quantify and rank human intelligence has been driven by nefarious motives. It has been driven by a desire to prove scientifically that certain groups of people are inferior to other groups of people.

There are no goalposts for this field, and there are no goalposts for the industry when they say that they are ultimately trying to recreate AI systems that would be as smart as humans. How do we even define what that means? And when are we going to get there if we do not know how to define the destination?

And what that effectively means is that these companies can just use the term *artificial general intelligence*, which is now the term used to refer to this ambitious goal to recreate human intelligence. They can use it however they want to, and they can define and redefine it based on what is convenient for them. So in OpenAI’s history, it has defined and redefined it many times.

When Sam Altman is talking with Congress, AGI is a system that is going to cure cancer, solve climate change, and cure poverty. When he is talking with consumers that he is trying to sell his products to, it is the most amazing digital assistant that you are ever going to have. When he was talking with Microsoft, in the deal that OpenAI and Microsoft struck, where Microsoft invested in the company, it was defined as a system that will generate $100 billion of revenue. And on OpenAI’s own website, they define it as highly autonomous systems that outperform humans in most economically valuable work.

This is not a coherent vision of one technology. These are very different definitions that are spoken out loud to the audience that needs to be mobilized to ward off regulation, or get more consumer buy-in into the industry’s quest, or to get more capital, more resources, for continuing on this journey with ambiguous definitions.

Speaking about different definitions through time, in 2015, in a blog post that Sam Altman wrote before OpenAI was officially announced, he explicitly outlined the existential risk by saying, “Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen, for example, an engineered virus, but AI is probably the most likely way to destroy everything.”

When Altman is writing for the public or speaking for the public, he does not just have the public as the audience in mind. There are other people that he is trying to motivate or mobilize when he says these things. And in that particular moment, Altman was trying to convince Elon Musk to join him on co-founding OpenAI. And Musk in particular was spending all of his time sounding the alarm on what he saw as a huge existential threat that AI could pose. So in that blog post, if you look at the language that Altman uses side by side with the language that Musk was using at the time, it mirrors all the things that Musk was saying.

> Identical. I mean, 10 years ago, Musk was going on podcasts, tweeting, whatever, that the greatest existential risk to humanity was AI.

> Yeah. And so, you know, his parenthetical there, “there are other things that we might actually be more likely to happen, like engineered viruses,” is because up until then Altman had been talking just about engineered viruses. And so now that he needs to pivot to speak to an audience of one, to Musk, he needs to kind of resolve the contradiction between what he is now elevating as his new central fear to be the same as Musk’s new central fear with what he had previously been saying. So that is why he is like, I think this is now, even though before I said this.

And are you saying that Sam Altman manipulated Musk? Because Elon did end up donating a huge amount of money to OpenAI and co-founding it, I believe, with Sam Altman.

Elon Musk did end up co-founding it with Altman. And certainly from Musk’s perspective, he does feel manipulated because he feels like Altman was engineering his language in a way that would make Musk trust him as a partner in this endeavor. And of course then Musk leaves. And through some of the documents that came out during the lawsuit that Musk and Altman are engaged in now, it has become clear that there was a degree to which Musk was actually muscled out a little bit. And so that is why he is left with this very intense personal vendetta against Altman, saying that somehow Altman tricked him into being part of this.

So in 2015, Sam Altman is writing these blog posts saying this is one of the greatest existential threats. At the same time, in 2015, Musk is doing some very famous speeches at the time at MIT. He said that AI was the biggest existential threat and compared developing AI to summoning the demon.

And what you are saying here is you are saying that Altman was just mirroring the language that Elon was using to get Elon involved in OpenAI. And later it appears, and again there is a legal case taking place now, that Sam might have muscled Elon out in some capacity.

> Yeah. So we know from the lawsuit and the documents that have come out in the lawsuit that Ilya Sutskever, who was the chief scientist of OpenAI at the time, and Greg Brockman, chief technology officer at the time, when they were deciding whether or not to maintain OpenAI as a nonprofit, because it was originally founded as a nonprofit, they decided, okay, we need to create a for-profit entity. But the question was who should be the CEO of this for-profit entity. Should it be Musk or should it be Altman? Because it is

### The fight over OpenAI’s founding leadership

They were the two co-chairmen of the nonprofit. In the emails, it became clear that Ilya and Greg first chose Musk to be the CEO. But through my reporting, I discovered that Altman then appealed personally to Greg Brockman, who was a friend of his that they had known, they had known each other for many years through the Silicon Valley scene, and said, "Don't you think that it would be a little bit dangerous to have Musk be the CEO of this company, this new for-profit entity, because, you know, he's a famous guy. He has a lot of pressures in the world. He could be threatened. He could act erratically. He could be unpredictable. And do we really want a technology that could be super powerful in the future to end up in the hands of this man?" And that convinced Greg, and Greg then convinced Ilya, you know, I think there's a point here. Do we really want to give this much power to Musk? And that is why Musk then leaves, because then the two switch their allegiances. They say, "Actually, we want Altman to be the CEO." And then Musk is like, "If I'm not CEO, I'm out."

> So, it sounds like Sam again managed to persuade someone to do something.

Mhm.

I guess this begs the question, what do you think of Sam Orman?

I think he's a very controversial figure.

You did an interesting pause. It's a pause where someone tries to select their words.

Well, this is what's so interesting about those interviews is people are extremely polarized on Altman there. No one has in-between feelings about him. Either they think that he's the greatest tech leader of this generation, akin to the Steve Jobs of the modern era, or they think that he's really manipulative and an abuser and a liar. And what I realized because I interviewed so many people is it really comes down to what that person's vision of the future is and what their goals are. So if you align with Altman's vision of the future, you're going to think he's the greatest asset ever to have on your side because this man is really persuasive. He's incredible at telling stories. He's incredible at mobilizing capital, at recruiting talent, at getting all the inputs that you need to then make that future happen. But if you don't agree with his vision of the future, then you begin to feel like you're being manipulated by him to support his vision even if you fundamentally don't agree with it. And this is the story especially of Dario Amodei, CEO of Anthropic, who was originally an executive at OpenAI. So for people that don't know, Dario now runs Anthropic, which is the maker of Claude. A lot of people probably are more familiar with Claude.

> Yeah. And it's one of the biggest competitors to OpenAI.

And Amodei at the time when he was an ex executive at OpenAI, he thought that Altman was on the same page with him and then over time began to feel that Altman was actually on exactly the opposite page of him and felt that Altman had used Amodei's intelligence, capabilities, skills to build things and bring about a vision of the future that he actually fundamentally didn't agree with. And so that's why people end up with this bad taste in their mouths. And so, you know, I've been covering the tech industry for over eight years and covered many companies. I've covered Meta, Google, Microsoft in addition to OpenAI, and OpenAI and Altman is the only figure that I've seen this degree of polarization with, where people cannot decide whether he's the greatest or the worst.

> You mentioned Dario there and I found it really what I found really interesting is to look at how people's quotes evolve over time with their incentives. So I was looking at all of the all of the things they've said on the record on podcasts in their blog post to see how it's evolved over time and Dario who was the former VP of research open AAI and has now moved on to anthropic who are taking a slightly different approach to developing AI said back in 2017 while he was still at open AI that this is a quote I think at the extreme end is the Nick Bostrom style of fear that an AGI could destroy humanity. I can't see any reason in principle why that couldn't happen. My chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%.

And also you mentioned Ilya who was a co-founder of OpenAI and then left. I guess the first question I'd ask is why did Ilya leave?

It's a great question. So he was instrumental in trying to get Sam Altman fired and he's another one of the people who over time began to feel like he was being manipulated by Altman towards contributing something that he didn't believe in. And for, you know, because I interviewed a lot of people, Ilya in particular had two pillars that he cared about deeply. One is making sure we get to so-called AGI and the other is making sure that we get to it safely. And he felt that Altman was actively undermining both things. He felt that Altman was creating a very chaotic environment within the company where he was pitting teams against each other, where he was telling different things to different people.

> Have you ever spoken to him?

I have. So I interviewed him in 2019 for a profile that I did of OpenAI for MIT Technology Review.

> And back in 2019, he has a quote where he says, "The future's going to be good for AIs regardless. It would be nice if it was also good for humans as well. It's not that it's going to actively hate humans or want to harm them, but it's just going to be so powerful. And I think a good analogy would be the way that humans treat animals. It's not that we hate animals. I think humans love animals, and I have a lot of affection for them. But when the time comes to build a highway between two cities, we are not asking the animals for permission. We just do it because it's important to us. And I think by default, that's the kind of relationship that's going to be between us and AI, which are truly autonomous and operating on their own behalf."

And that was in 2019, the year that you interviewed him.

One of the things that I feel like we should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we mean by intelligence? And a huge part of the views of the different people and the quotes that you're reading derives from a specific belief that they each have in this question of what is intelligence, what constitutes intelligence. For Ilya, he has throughout his research career felt that ultimately our brains are giant statistical models. This is not something that, you know, we actually know, but this is his own hypothesis, also the hypothesis of his mentor Geoffrey Hinton, who also was on this podcast. This is why they have such a strong conviction in the idea of building AI.

### AI as an imperial project

The argument rests on a hypothesis: that these are statistical models, and that this particular approach will lead to intelligent systems in the same way that we are intelligent. That is a hypothesis they hold. It has not been proven by science, and some people vehemently disagree with it.

But if you step into their shoes and take on that hypothesis, and assume it is true — that our brains are in fact statistical engines, and that these systems they are building are also statistical engines, made bigger and bigger and bigger until they become the size of the human brain — then the comparison they make starts to make sense inside their framework. That is why they say the system will become equal to human intelligence and then maybe exceed human intelligence.

Ilia gave a talk at one point at a very prominent AI research conference held every year called *Neural Information Processing Systems*. It is a mouthful, but he gave this keynote in which he showed a chart of the size of brains and the intelligence of a species, and it was roughly linear: the bigger the size of the brain, the more intelligent the species. For him, that means he thinks he is building a digital brain, because he thinks brains are just statistical engines. So from that logic it is: if we build a bigger statistical engine than the human brain, then based on this chart it will be more intelligent, and then we will be subjected to the same treatment that we have subjected animals to.

But it is really important to understand that these are scientific hypotheses held by specific individuals within the AI research community. There is a lot, a lot of debate about whether this is in fact the case, and some of the biggest critics say it is very reductive to think of our brains as simply statistical engines.

> Why does it matter to know the mechanism? Is it not just important to know the outcome, which is that it is going to be able to make a video for me, or agents are going to be able to do the work that I do? Does it really matter for us to know the mechanism behind it?

Yes and no. It matters because these companies are driving their future actions based on this hypothesis. They have decided, *we think this hypothesis is true; we should just continue building larger and larger statistical models in the pursuit of artificial general intelligence.*

And that is having global consequences. In order to continue doing that, they are hoovering up more and more data. They are building more and more data centers. They are exploiting more and more labor in order to continue on this path.

Here is a question that I think is important to ask: why are we trying to build AI systems that are duplicative of humans? We are kind of having this conversation right now where we have just taken the premise of this industry as a good thing. They said that we should be building AGI, so we say that we should be building AGI.

I would like to ask: why are we doing that? Why is it that we are building a technology that is ultimately designed to replace and automate people away? That is not the enterprise of technology. We should be building technology, and the purpose of technology throughout history has been to improve human flourishing, not to replace people.

This is a critical part of my critique of these companies and these scientists, who have just adopted this goal and have relentlessly pursued it, and have had enormous capital and enormous resources to pursue it. Is this the right goal? Why are we doing this? Why can’t we just build AI systems that do things like accelerate drug discovery and improve people’s health care outcomes, which are systems that have nothing to do with the statistical engines they are trying to build to duplicate the human brain?

> So why are they doing it? I mean, you have interviewed all these people. I think it is 300 people in total, 80 or 90 of them from OpenAI, the maker of ChatGPT. Why do you think they are doing it?

I think it is because they are driven by an imperial agenda. And that is why I call these companies empires of AI.

> What do you mean by an imperial agenda? What does that term mean?

Empire is the only metaphor I have ever found that fully encapsulates all of the dimensions of what these companies do, the scale they operate at, and what motivates them to do what they do. There are many parallels between what I call the empires of AI and the empires of old.

They lay claim to resources that are not their own in the pursuit of training these models: the data of individuals, the intellectual property of artists, writers, and creators. They engage in land grabbing in order to build these supercomputer facilities for training the next generation of models.

Second, they exploit an extraordinary amount of labor. They contract hundreds of thousands of workers all around the world, including in the United States, to ultimately make these technologies. We can talk about that more. They also design their tools to be labor-automating, so that when the technologies are deployed, it also affects labor rights because it erodes labor rights. This is a political choice that they have.

Third, they monopolize knowledge production. They project the idea that they are the only ones who really understand how the technology works, and so if the public does not like it, it is because they do not actually know enough about this technology. They do this to the public. They do this to policy makers. And they have also captured the majority of the scientists who are working on understanding the limitations and capabilities of AI.

### How AI companies shape research and silence critics

"You think they're gaslighting the public in a way?"

"They are. Yeah. So if most of the climate scientists in the world were bankrolled by fossil fuel companies, do you think we would get an accurate picture of the climate crisis?"

"No."

"And in the same way they employ and bankroll the AI industry employs and bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways simply by funneling money to their priorities so that only certain types of AI research are produced. But they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timmy Gabru in my book who was the ethical AI team co-lead at Google when she was literally hired to critique the types of AI systems that Google was building. She then co-wrote a critical research paper that was showing how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gabru and then fired her other co-lead Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda."

"Did you have an example where this is happening to journalists as well that are asking questions of their team members? I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages, and this person was from one of the big AI companies."

"This was opening. I started subpoenaing some of its critics. Yeah. As part of a campaign of intimidation, but also what appeared to be a campaign of fishing for more information to figure out, to map out the network of critics further. But this was a man who runs a small watchdog nonprofit and they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion. But during the period where it was sort of existential for OpenAI to complete this conversion, there were a lot of civil society groups and watchdog groups like MIDAS who were trying to prevent the process from happening in the dead of night. They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that there was a knock on his door and he was served papers."

"What did the papers say?"

"The papers asked him to reproduce every single piece of communication that he had had that might have involved Musk. So this was like this strange paranoia that OpenAI had that Musk was somehow"

### The empire narrative behind AI

#### The empire story around AI

Empires always have a story they tell the public: *we are the good empire*, and we need to be an empire in the first place because there are also bad empires in the world. The pitch is that if you allow us to take all the resources and use all of the labor, then we promise we will bring you progress and modernity for everyone.

The promise is a kind of utopic state, almost like an AI heaven. But if the evil empire does it first, then we will descend into a hell.

In this framing, the evil empire is most often China. But in the early days, OpenAI evoked Google as the evil empire. The decisions were all about *we need to do it first*, because otherwise Google — this evil corporation driven by profit — would win, while OpenAI would be the benevolent nonprofit. It was presented as a critical contest of who wins.

#### What the builders believe

Asked whether the people building these AI companies believe the outcome is going to be all good, or whether they think it is going to serve everyone and create the age of abundance where everything goes up well, the answer was that this is a core part of the mythology they create around the AI industry.

That mythology includes the belief that it could go very badly. It goes hand in hand. They need that part of the myth in order to then say: *and that is why we need to be in control of the technology*, because that is the only way it is going to go really, really well.

Altman has said publicly that the worst case is *lights out for everyone*, but the best case is that we cure cancer, solve climate change, and there is abundance. Dario Amodei has used the same kind of rhetoric: worst case catastrophic or existential harm for humanity, best case mass human flourishing.

Those are two sides of the same coin. They need both narratives in order to keep justifying an extremely anti-democratic approach to AI development, one where there should not be broad participation in developing the technology. They must be the ones controlling it at every step of the way.

### The OpenAI book dispute

#### The books coming out about OpenAI

“There are some books coming out about OpenAI and me. We only participated in two of them: one by Kesh Hegy, focused on me, and one by Ashley Vance on OpenAI.” He went on to say that no book will get everything right, especially when some people are so intent on twisting things, but that these two authors are trying to. You quote-retweeted that tweet from Sam Altman and said, “The unnamed book *Empire of AI* is mine.”

Do you believe that tweet from Sam Altman was in reference to your book?

> “100%.”

Because there are only three books coming out about him, and he had caught wind that your book was coming out?

He knew my book was coming out because I had contacted OpenAI from the very beginning of my process and said, “I’m working on a book now. Will you participate in it?” And actually, initially they said yes, even though my history with OpenAI was already complicated.

#### The history with OpenAI

I profiled the company for *MIT Technology Review*. I embedded within the office for three days in 2019. My profile came out in 2020, and the leadership were very unhappy. In my book, I actually quote an email that I received that Sam Altman sent to the company about my profile, saying, “Yeah, this is not great.”

From then on, the company’s stance toward me was: “We are not going to participate in anything that you do. We are not going to respond to anything, any of the questions that you receive.” And this was explicitly articulated. It wasn’t me inferring it.

I had a colleague at *MIT Technology Review* who also covered AI. At one point, OpenAI sent him a press release and said, “We would love for you to cover this story.” He said, “I’m really busy. Will you send it to Karen?” And they were like, “Oh, no. We have a history. You understand?”

So for three years they refused to talk to me. But then I ended up at *The Wall Street Journal*, where they felt a bit compelled, because it was the *Journal*, to reopen the lines of communication. I started having more dialogue with them. Every time I wrote a piece, I would always send them my request for comment. I would always ask them, “Will you sit for interviews?” And we did get to a more productive relationship.

Then I embarked on the book. I left the *Journal* to focus on the book full-time, and I told them right away, “I’m working on this book. I want to continue this productive conversation where I make sure I reflect OpenAI’s perspective in the book.” They were like, “We can arrange interviews for you. You can come back to the office. We’ll set up some conversations.” And then, as we were going back and forth...

### Access denial and interview pressure

On this, the board fired Sam Altman. And that is when things started going kind of south, because the company started becoming very sensitive to scrutiny. They started pushing, kicking the can down the road, down the road, down the road. The speaker kept saying, **“Hey, when are we rescheduling this? What’s going on?”** Then an email came saying, **“We are not going to participate at all. You are not coming to the office. You’re not doing interviews.”**

The speaker had already booked the tickets and was already going to fly to San Francisco to do the interviews. So the response was, essentially, that was fine: they would still engage in the process by giving extensive requests for comment, asking through reporting, and keeping the company updated on everything being found so that it could still choose to comment. The speaker gave them **40 pages of requests for comment** and gave them **over a month** to respond to all of it. This was the period when the tweet came out, while all of this back and forth was happening, and that is when Altman tweeted this.

> H

And they never responded to a single one of the 40 pages.

Altman does a lot of interviews. He is doing a lot of interviews all the time. He has done every podcast. The speaker has seen him on everything from **Tucker Carlson** to, it is believed, **Theo** and **Joe Rogan**, and on podcasts all over the world.

The question is: why will he not do this one?

The reply is that maybe he just will not. The speaker does not know why. They do not know. They think they are fair with everyone. They just ask questions they genuinely care about. They do not come in with huge preconceptions, at least when meeting people for the first time. But they have heard through the grapevine that he does not want to do theirs.

Returning to what was said earlier about the way OpenAI and these companies control research: the question was whether they also do this with journalists. The answer is yes. Apparently, they also do it with anyone who has a broad mass communications platform.

It is not just about the conversation you are going to have with them. It is about who else you choose to platform. There is a huge problem in technology journalism where companies know that a really big carrot they can give to technology journalists is **access**.

> Yeah. Yeah. Yeah.

And they will withhold that access at the drop of a hat if they catch wind that you are speaking to someone they did not want you to speak to.

The speaker says this is very true, and that the average person does not really truly understand it. It may sound like theory, but they are not going to name names because they do not think it is important. There is, however, a particular person in AI whose team has basically dangled the carrot of them coming here for about **18 months**.

The speaker says they do not need the carrot dangled. They are going to speak to whoever they want regardless of the carrot or not. And if that person comes, if they want to come, they will get a fair shot. They will be asked all genuinely curious questions about what they are doing and about their incentives. They will not be gotcha'd. The speaker says they do not have a history of ever gotchaing anybody. Even if they disagree, even if they have a different opinion, they will ask the question.

But the carrot gets dangled, and the pitch is: if they are thinking about it, then let us think about a date. The strategy, as the speaker sees it, is that if you just dangle it for long enough, then those people will perform in the way you want them to, and they will be pleasant about you. They will not be critical. They will not give a criticism.

> Our critics.

A lot of the game, the speaker says, is simply to dangle the carrot forever. That, they argue, is the optimal outcome: just keep dangling it, keep saying they are looking at the schedule.

It just does not work. In the modern world, you just have to go there, give your opinion, and allow the conversation to happen.

### Board conflict and image management

clash of ideas in the public forum, let the viewers decide for themselves.

> Yeah.

This is such a huge part of their machinery: the way they use these tactics to massage the public image of these companies and make sure that information they do not want out, and even opinions they do not want out there, go out there.

> Mhm.

This is why I feel very lucky now that opening I shut the door early on me. At the time I did not feel lucky. I felt like I had screwed myself over. I was nicer access to a journalist, right? Like you are supposed to report the truth and you are always supposed to report in the interest of the public. Like that is the point of journalism. And in that moment, I was relatively junior in my career. I was like, did I misunderstand what journalism is about? Should I have actually been playing the access game?

> Mhm.

But it was too late. I had the door shut to me, and so I had to build my career understanding that the front door was never going to be open.

> Yeah.

That actually really strengthened my own ability to just tell it like it is, like objective. Yeah. And just report what I see are the facts being presented to me, irrespective of whether the company likes it or not. And most often the company really does not like it, but I can continue to do the work. They do not need to open the front door for me. I was still able to do more than 300 interviews.

So Sam Altman gets kicked off the OpenAI executive team. Did you find out why that happened?

> Yeah, there is a scene-by-scene recounting.

From who? I cannot remember the exact number of sources, so I do not want to misquote myself, but it was around six or seven people that were directly involved or had spoken to people directly involved in the decision-making process. So, Ilya Sutskever is seeing these serious concerns about the way that Altman’s behavior is leading to bad research outcomes and poor decision-making at the company. He then approaches a board member, Helen Toner. Ilya, for anyone that does not know, is the co-founder we mentioned earlier. The co-founder of OpenAI we mentioned earlier.

> Yes.

And he kind of does a bit of a sounding-board thing to Helen just because Ilya is freaking out. He has been sitting on these concerns for a while, and he is like, if I tell this to someone, this could also be really bad for me if Altman finds out. And so he asks for a meeting with Toner, and in that first meeting he barely says a thing. He is just dancing around, trying to figure out, hey, is this someone that I can maybe trust to divulge more information.

And Toner’s role and responsibilities at OpenAI were?

> She was a board member.

Just a board member.

> Yeah. And specifically an independent board member. So OpenAI, when it was a nonprofit, the board was split between people who had a financial stake in the company and then people who were fully independent, and this was meant to be a structure that would balance the decision-making to be in the benefit of the public interest rather than to be in the benefit of the for-profit entity that OpenAI then created.

And Ilya, as a non-independent board member, was approaching Toner as an independent board member to try and see whether or not she was potentially seeing or hearing the same things that he was about the effect that Altman was having on the company. This then sets off a series of conversations, first between Ilya and Helen and then between Emmett Shear and some of the board members. Samir Murati was at that point the chief technology officer of OpenAI, where these two senior leaders, essentially through these conversations and through documentation that they are pulling together, like email, Slack messages, and so forth, they convey to the independent board members that they are very concerned about Altman’s leadership, like he is creating too much instability at the company and it is like he is the root of the...

### Why Altman was pushed out

The problem, as they saw it, was not just that people were working hard. The claim being made to the independent board members was that the problem would not be fixed unless Altman was removed, because of the way he was pitting teams against each other and creating an environment where people were unable to trust each other anymore and were competing rather than collaborating on what was supposed to be this very, very important technology.

When you say instability, that is a quite vague term. It could mean lots of things. Instability could mean pushing people hard to work harder, right? What do you mean by instability in as specific terms as you can possibly say them?

When ChatGPT came out in the world, OpenAI was wholly unprepared. They did not think that they were launching a gangbusters product. They thought they were releasing a research preview that would help them get the data flywheel going, collect a bunch of data from users that would then inform what they thought would be the gangbusters product, which was a chatbot using GPT-4, and ChatGPT was using GPT-3.5.

Because of that, there were servers crashing all the time, because they had to scale their infrastructure faster than any company in history. There were outages. They were also trying to hire faster than any company in history to have more personnel there, and sometimes they were hiring people and then realizing, "Actually, we made a mistake. We should not have hired you." So they were firing people left and right, and people were just disappearing off of Slack, and that is how their colleagues would learn that they were no longer at the company.

So, yes, like many fast-growing companies, it was a very chaotic environment, and a particularly chaotic environment because it was extra fast. They had to accelerate more than any other startup. On top of that, Mira Murati and Ilya Sutskever felt that Altman was making it worse. He was not actually effectively ameliorating the circumstances of the chaos. He was actually sowing more chaos, getting these teams to be more divided.

This is where it is important to understand that the executives and the independent board members were all operating under the idea that they were building AGI, and that AGI could either be devastating or utopic to humanity. So it is not, yes, like any other company, and no, not like any other company. In their view, you cannot have this degree of chaos as the pressure cooker for creating a technology that could make or break the world.

That is basically what the independent board members also began to reflect on. They had conversations among themselves where they were like, "Well, based on what we are hearing about Altman's behavior, if this was an Instacart, would that warrant firing him?" And they concluded, "Maybe not, but this is not Instacart." That is why they were like, "Well, crap. Maybe this actually does rise to the bar where we should consider replacing him, because we are ultimately building a technology that we think could have transformative impacts either in the positive or negative direction."

That is what happens. It is these two executives and then the independent board members, and they were also hearing other feedback from their connections within the company and from other people in the industry. At one point, Adam D'Angelo, who is one of the independent board members and the CEO of Quora, which is a tech startup in the Valley, is at a party in San Francisco, and he starts to hear some of these rumors that there is something weird about the way that OpenAI has structured its OpenAI startup fund, which was the fund the company had created to start investing in other startups.

He realizes they had never really seen documentation about how the startup fund had been set up from Altman. Finally they get the documents, and it turns out that OpenAI startup fund is not OpenAI's startup fund. It is Altman's startup fund. That was one of several experiences the independent board members were also having, where they were like there is something not right about the fact that there continuously are inconsistencies between the way that Altman is portraying what is being done versus what is actually being done.

When these two executives approach the board, or the independent board members, then they are like, "Okay, this lines up with also the experiences that we have been having." At that point, they then have this series of very intense discussions where they are meeting almost every day, talking about whether they should actually really consider removing Altman. In the end they conclude yes, they should. And if they are going to do it, they need to do it quickly, because they were very concerned that the moment Altman found out, his persuasive abilities would make it impossible to do.

So they end up firing Altman without telling anyone. They do not talk to any stakeholders to get them on the same page. Microsoft gets a call right before they execute the action saying, "We are going to fire Altman." And Microsoft, for anyone that does not know, is a lead investor in OpenAI at the time.

Yes, one of the only investors in OpenAI at the time. And that is what then devolves the whole thing, because every single person that is affected by this decision is now extremely angry that they were not involved. That is what then creates this campaign to bring Altman back, and then Altman is reinstalled as CEO days later.

### A sponsor read on productivity and board power

Whisper Flow is going to create such a huge productivity advantage for you. Whisper Flow is an app that you can get on your computer and on your phone, on all your devices, and it allows you to speak to your technology. So instead of writing out an email, you click one button on your phone and you can just speak the email into existence, and it uses AI to clean up what you were saying. Then when you're done, you just hit this one button here and the whole email is written for you. It is saving me so much time in a day because Whisper learns how I write. On WhatsApp, it knows how I am a little bit more casual. On email, a little bit more professional. And also, there is this really interesting thing they have just done: I can create little phrases to automatically do the work for me. I can just say Jack's LinkedIn, and it copies Jack's LinkedIn profile for me because it knows who Jack is in my life.

This is saving me a huge amount of time. This company is growing like absolute crazy, and this is why I invested in the business and why they are now a sponsor of this show. Whisper Flow is frankly becoming the worst-kept secret in business, productivity, and entrepreneurship. Check it out now at `whisperflow` spelled `w i s p r l o w.ai/steven`. It will be a game changer for you.

There is a phase a lot of companies hit where they are no longer doing the most important thing, which is selling, and they get really bogged down with admin. It is often something that creeps up slowly and you do not really notice until it has happened. Slowly, momentum starts to leak out. This happened to us, and our sponsor Pipe Drive was a fix I came across 10 years ago. Ever since, my teams across my different companies have continued to use it.

Pipe Drive is a simple but powerful sales CRM that gives you visibility on any deals in your pipeline. It also automates a lot of the tedious, repetitive, and time-consuming parts of the sales process, which in turn saves you so many hours every single month, which means you can get back to selling. Making that early decision to switch to Pipe Drive was a real game changer, and it has kept the right things front of mind. My favorite feature is Pipe Drive's ability to sync your CRM with multiple email inboxes so your entire team can work together from one platform. And we are not the only ones benefiting: over 100,000 companies use Pipe Drive to grow their business. So if something I have said resonates, head over to `pipedrive.com/ceeo` where you can get a 30-day free trial. No credit card or payment required.

How does a CEO of a major company get fired by the board? Because board members, there is a quote in your book on page 357 where you say about Ilia

### Rivalries, status, and who should hold power

saying, "I don't think Sam is the guy who should have the finger on the button for AGI."

I asked myself this question. I work with lots of people here. We have 150 people that work in this business, and those people know me best. They see me on camera. They see me off camera. So if they said that, *we don't think Steven is the right person to host the diary...* it would take a lot for them to say that. They must have seen something off camera for them to go, *we don't think he's the right person to be on camera.* Or for whatever reason.

And in the case of AI, which is much more consequential than a podcast that is filmed in my old kitchen, it almost sends a chill down one's body to think that the co-founder of a business has gone to the board and said, *this isn't the guy to lead this consequential thing.*

Mira Murati then also said, "I don't think Altman is the right guy," and then they both left later. So then Altman comes back, and lo and behold, Ilya never comes back. So his concerns about the fact that Altman finding out would be bad for him manifested. He ended up not coming back, and Mira Murati then left shortly thereafter.

Quite a lot of these people leave, don't they? OpenAI.

They do. If you consider one of the origin stories of OpenAI, it is this dinner that happened at the Rosewood Hotel, which is a very swanky hotel right in the heart of Silicon Valley, and that was one of Elon Musk's favorites whenever he was coming up from LA to the Bay Area. There was this dinner there where Altman was intending to recruit the OG team that would start OpenAI. So he's kind of telling everyone, *you might have a chance to meet Musk because Musk is going to come to this dinner.* He cold emails Ilya and gets Ilya to come because Ilya specifically wants to come because he wants to meet Musk. He also emails all these other people, including Greg Brockman, Dario Amodei. These are all people that ended up working at OpenAI, and almost all of them end up working at OpenAI and leaving. Almost all of them end up leaving specifically after they clash with Altman.

And Ilya, he left and launched a company called Safe Super Intelligence.

Yeah.

Which is, I mean, that's an indirect if I've ever heard one. Do you know what I mean? If someone co-founded this podcast with me and then they left and started a podcast called Safe Podcasting, I'd take that as a slight. I'd have people knocking on their door and asking for their texts.

One of the things that is happening here is it is not a coincidence that every single tech billionaire has their own AI company.

They want to create AI in their own image, and that's why they keep not getting along. In fact, it's not just that they don't get along, they end up hating each other after working together, and then splinter off into their own organizations.

So after Musk leaves, he starts xAI. After Dario leaves, he starts Anthropic. After Ilya leaves, he starts Safe Super Intelligence. After Mira leaves, she starts Thinking Machines Lab. They want to have control over their own vision of this technology, and the best way that they have derived from their experiences of trying to put their vision into the arena is by creating a competitor and then competing with OpenAI and with all the other companies out there.

Do you think some of these AI CEOs realize that they are quite literally summoning the demon, as Elon said 10 years ago, but they don't really care because being the person that summoned the demon makes you consequential and powerful and historical, even if the outcome is potentially horrific? Even if there's like a 20% outcome of it being horrific?

I remember, I think it was Dario, he's the one that said there's somewhere between a 10% and 25% chance of things going catastrophically wrong on the scale of human civilization. 25% is a one in four chance. If you put bullets in a four-chamber revolver and said, "Steven, the upside is you could become a multi-gazillionaire and be remembered forever. The downside is that there would be a bullet in your head," there is no chance that I would take that bet with a 25% potential chance of things going catastrophically wrong.

So, I have a very long answer to this because do they know if they're summoning the demon? It really depends on what we define as summoning the demon. In this particular case, to go back to what we were saying before, there's a mythology that the AI industry uses where summoning the demon is an integral part of convincing everyone that therefore they can be the only ones that are developing this technology.

I got it. So on one end, you got to say, if we don't, China will, and that's terrible.

Yeah. But if we let anyone else do it other than me, then we're as well.

Exactly.

So that means that I have to do it and you have to give me money and support.

Exactly. So when they're saying these things, we should understand it as not as like a genuine prediction based on what they're seeing, because first of all, we don't predict the future. We make it. We should understand this as an act of speech to persuade other people into believing that they should seed more power, more resources to these individuals. And so, do they know that

### AI mythmaking as a source of influence

They are **summoning the demon**? I mean, they are purposely trying to create this feeling within the public that they are, because it is a crucial part of their power. But do they, if we were to define it plainly, realize that the things that they are doing are already having really harmful impacts all around the world on vulnerable people, vulnerable communities, and vulnerable countries? That is where I am like, maybe yes, maybe no. And they do not really care because, in the frame of mind, I sometimes use the analogy that the AI world is like *Dune*.

> *Dune* for anyone that does not know *Dune*.

*Dune* is a science fiction epic written by Frank Herbert, and it is set in this intergalactic era where there are all these houses and they are fighting each other for spice. So it is a callback to colonialism and empire, and they are all trying to control the spice. But one of the features of this story is that there are these myths that are seeded on the different planets about a religious myth, basically, about the coming of the Messiah that are used as ways to control the people.

And Paul Atreides, when he arrives at the planet Arrakis, with the intention of trying to then fight against the empire and avenge his father’s death, steps into a myth that has been seeded on this planet that says that one day there will be a Messiah that comes and saves the planet. So he steps into the role of the Messiah and leans into this idea in order to better control the people and rally them behind him as a leader to help with this quest.

He knows that it is a myth in the beginning, but because he lives and breathes and embodies it, it kind of starts to blur in his mind whether this is really a myth or whether he is really the Messiah. And this is what I think happens in the AI world. On one hand, there are all these executives that actively engage in mythmaking because, you know, I have all these internal documents that I write about in the book where they are very keenly aware of how to bring the public along with them by showing them dazzling demonstrations of the technology, by crafting a mission that will sound really good and make people give more leniency to their companies.

So they know they are doing the mythmaking, and also I think many of them lose themselves in the myth because they have to live and breathe and embody it day in and day out. And so when, you know, Dario says he thinks that 10 to 25% of the future could be catastrophic, or whatever the probability is, 10 to 25%, he is actively engaging in the mythmaking, but also he is losing himself in the myth. Like, I think if you were to ask him, "Do you genuinely believe that?" he would be like, "Yes, I..."

### Cognitive dissonance in AI leadership

Because there has been a blurring of when he is saying something just to say something versus when he actually believes what he is required to believe in order to continue doing the things that he is doing.

> **And this is the whole psychology of cognitive dissonance, right?** The brain struggles to hold two conflicting worldviews at the same time. So it is incentivized, or it endeavors, to dismiss one. If you wanted to be a healthy person but also a smoker, and I pointed out that smoking is bad for you, the first words out of your mouth are going to be, **“Yes, but…”**

#### Fundraising, messaging, and moral branding

I kind of see that at the moment because these companies have to raise extortionate, huge amounts of money to fund their AI research, and they are building out all of these data centers. So when they are out in the public, they are always fundraising. All of these major companies are fundraising all the time at the moment.

So you cannot be fundraising and saying, **“I’m going to destroy your children’s future potentially. There’s 25% chance that your children aren’t going to have a great life.”** Which might be the truth. I mean, that is actually what they say. Dario does this famously. He is like—he does that—but the others, Sam is not doing that as much anymore.

Yes, and it is because it goes back to how each of them distinguishes themselves a little bit as the brand that they need to project.

Do you think any of them have a stronger moral compass than others? Because I think Dario often gets the credit for having more of a backbone and being more conscious of implications.

He does get a lot of credit for that. He is from Claude and Anthropic, for anyone that does not know.

I do not think it truly matters, that question, the answer to that question, because to me even if you were to swap all the CEOs for someone that people would say is better at running these companies, it does not fix the problem that I identify in the book, which is that there is a system of power that has been constructed where these companies and the people running these companies get to make decisions that affect billions of people’s lives around the world, and those billions of people do not get any say in how it goes.

Those people, they can go to the polls, right? So if the public are sufficiently educated, they can go to the polls and pick a leader that says they are going to legislate, or pass laws, or try and pass laws.

Yes. But at the speed and pace at which these companies operate, and at the sheer scale and size, they are able to also spend extraordinary amounts of money—hundreds of millions in this upcoming midterms—to try and kill every possible piece of legislation that gets in their way and craft legislation that would codify their advantage.

And so to me, I think sometimes as a society we obsess a little bit with whether these leaders are good or bad people. And to me the bigger question is whether the governance structure that we have created is a sound one, or one that allows broad participation, or an anti-democratic one that has consolidated this decision-making power in the hands of the few. Because no person is perfect.

I do not care who is on top of these companies. They are not going to have the ability to make decisions on behalf of so many people around the world who live and talk and have a culture and history that are fundamentally different from them without things going wrong. And so that is why throughout history we have moved from empires to democracy. It is because empire as a structure is inherently unsound. It does not actually maximize the chances of most people in the world being able to live dignified lives.

#### The argument for acceleration

I am going to try and take on their point of view. So this is me playing devil’s advocate. But Karen, if the US do not continue to accelerate their research with AI, at some point China’s model is going to become so smart and intelligent that we are basically going to have to rent it off them and we are going to be, you know, they will get the scientific discoveries. They will discover the new era of autonomous weapons and we will be their backyard. And logically, that argument does appear to be pretty true.

No, it is not.

If we scale up, if we just imagine any rate of change with this intelligence, at some point we are going to come to a weapon that could theoretically disable all of the United States electricity, their weapons systems. It would know exactly how to disable the United States from a cyber perspective because it would be that smart. All you have got to imagine is any rate of improvement over any long period of time. So this is a theory that might be true, and if it is true—

I mean, yes, any theory might be true.

But if—going to this point of, even if it is a small percentage, it is worth paying attention to on the other side of the foot. This is a theory that people talk about. It could be the case that the most intelligent civilization is going to be the superior civilization. Logically, that is a pretty sound thing to say.

No.

So there are a lot of fundamentals in this argument that would need to be true in order for this to be a viable argument. And let us knock them down one by one. So the first one is that these systems are intelligent and that just scaling them is going to bring us more intelligence. So far so true.

No, it is actually not, because first of all, again, we do not actually know if these systems are—intelligence is not the right analogy almost. It is sort of like, is a calculator a calculator? Can do math problems faster than a human. Does that make it intelligent?

It has a narrow intelligence because they are solving a narrow problem, which is like 1 plus 1 equals 2. But these systems, they actually also are quite narrowly intelligent in the sense that even though these companies say that they are everything machines that can do anything for anyone, they actually can only do some things for some people. This is like the jagged frontier of these AI models: some of the capabilities are quite good, other capabilities are not that good.

You know why that happens? It is because the company can only focus on advancing certain types of capabilities. It cannot literally focus on advancing all types of capabilities. They have to actually set their mind to advancing a certain capability by gathering the data that is needed for that capability, by getting a bunch of human contractors to annotate and train the model to do that exact thing.

And so scaling these models is actually a perpendicular question to whether we are actually getting more cyber capabilities specifically and more military capabilities specifically.

I would argue that most of the...

### The assumptions behind intelligence claims

The top people in AI believe that intelligence is going to continue to scale for some time. A lot of them do, like Geoffrey Hinton does.

> And again, it's back to his hypothesis about how human intelligence works and what the appropriate model of the brain is. His hypothesis throughout his career has been that the brain is a statistical engine.

But that is his hypothesis, and that is not universally agreed upon, especially among people who are not in the AI world. When you talk with neuroscientists and psychologists, people who actually study human intelligence in the human brain, that is where you start to get a lot of debate and disagreement about this particular view that Hinton has.

This is one of the things: AI is already being used in the military and has been used in the military for a long time. But specifically accelerating large language models is not the only path for getting military capability. The companies would have to choose to specifically pick military capabilities to accelerate, not just general intelligence. It is like, you know what I'm saying? They create this myth that they are actually pushing the frontier of all of the capabilities of the model, but that is not what is actually happening internally.

I had hundreds of pages of documents on how they were specifically training models. They pick what capabilities they want to advance, and you know how they pick them? It is based on which industries and countries would be able to pay them the most money for their services. So they pick finance, law, medicine, healthcare, commerce. It is not actually intelligence like a baby, where the more the baby grows up, the more it starts having these general abilities.

> I think I have jagged intelligence. I'll be honest. I wasn't going to say it, but I think I know a little. No, I know a lot about a little bit.

Yes, but you also have the capability to learn and acquire knowledge by yourself. And you also have the ability to choose what you are going to learn and acquire by yourself.

It is not easy, and it takes a lot more time than these models. It seems less compute, but you can learn how to drive in one place and then immediately know how to drive in another place. These models cannot do that. Every time a self-driving car is shifted to another location, it has to completely retrain on that location.

It is like all the self-driving cars. I mean, we are sitting in Austin right now, and there are all these self-driving cars driving through Austin. But when one of them learns, they all learn.

> Which is...

Well, it is just because it is an operating system that has an AI model as part of it, and you are training the AI model and then you deploy that AI model across all the self-driving cars.

A big advantage is that if one Optimus robot learns one thing in one factory, they all learn it. Imagine that. Imagine if humans, if we all learned what all the other humans learned, that would give us such an unbelievable competitive advantage. I mean, one of the ways we did that is through communication.

They could not, because they could be learning the wrong thing, which has also happened again and again with these technologies: all of them then learn the wrong thing, and they all have the same failure mode. Part of the resilience of human society is that we do have different expertises and we also have different failure modes.

I think sometimes we hold AI models to a higher standard than we hold humans to. In a weird way, I hear on stage, we are in Austin at the moment, and I'd hear people go, "Ah, but you know, those AI models, they hallucinate sometimes." I'm like, "Have you met a human?" I hallucinate all the time. I can barely spell or do math.

Yes, but once again this is using this analogy that was specifically picked in the early days of the field as a way to market these technologies. We are repeatedly using the intelligence analogy and relating these machines to human intelligence as a way to try to gauge whether or not it is good or worthy or capable in society.

I think the output is the thing that is the most consequential, which is: okay, it might have a different brain and a different system, but does it arrive at the same capability? Is it able to do surgery on someone's brain? Is it able to drive a car? My car drives itself in Los Angeles. I do not touch the steering wheel, and I can drive for many, many hours. And here in Austin, I just saw the ones the other day where they've removed the steering wheel and the pedals, the new cyber cabs.

So I go, it does not really matter if it is using a different system if it is navigating through the world as a car. It has a better safety record than human beings. Then, as far as I am concerned, intelligence or not, it is like...

> Yes, you know,

but that was not the original argument that you made, which was that these systems are just generally going to become more intelligent across different things based on the prediction. This is a prediction that you are making, right? Like that. And this is a prediction that all the AI...

Ilia is making, Dario is making, Elon is making, Zuckerberg is making, Mann is making, Dennis is making.

And do you know what the common feature of all of them is? They profit enormously off of this myth.

Elon has recently spearheaded the construction of Colossus, a massive supercomputer in Memphis housing 100,000 GPUs specifically to scale up their API models faster than their competitors. It appears that they have all converged around this idea that you can brute-force your way to greater, more generalized intelligence. They have converged around the idea that you can brute-force your way into models that they can sell to people for automating certain tasks that are financially lucrative.

And I heard Elon say that if you are a surgeon, there is just no point. He was like, do not train to be a surgeon. He says in a couple of years' time, Optimus and AI generally are going to be better than any surgeon that has ever lived.

> Yeah. You know,

do you think these things are true? Well, you know, I am pretty sure it was Hinton that famously, slash infamously, said there would be no need for radiologists anymore.

There would be no need for radiologists anymore, and he set a deadline that we have already passed. I do not remember how many years. Radiology is doing great as a profession.

Do you think it will be in 5 years?

Okay. So this once again goes back to this question of why do we build technology, and why should we specifically be building AI? For me, the whole project of technology development and advancement is not to advance technology for technology's sake.

It is to help people. And there have been lots of research that has shown that actually the best

### Limits of autonomy and AI panic

#### Self-driving cars and statistical engines

Outcomes for people in a healthcare setting are best when the radiologist has the AI model in their hands and the human expert uses the AI model as a tool, as an input into their judgment. It is that combination that leads to the most accurate and early diagnoses of certain types of cancer, which then help improve the patient’s prognosis.

> Do you believe that in the coming years all the cars, pretty much all the cars on the road, will be driving themselves?

No.

> You don’t think so?

No.

> How come?

Because of the way the technology works.

These are statistical systems. The current way AI models are primarily developed is as statistical engines. You have what is called a neural network, which is a piece of software that has a bunch of densely connected nodes and parameters.

> Is this what they call parameters?

Yeah, pretty much. And you are pumping a bunch of data into it, and then it is analyzing the data and creating all of these correlations and patterns in the data. Through those patterns, the machine is then able to act autonomously.

The way they train a self-driving car is by recording all this footage and then having tens of thousands or hundreds of thousands of human contractors literally draw around every single vehicle in the footage, every single pedestrian, every single traffic light, every single lane marking, and label it exactly as such. Then that is fed into an AI model that can identify all of these different components, and then it is connected to another piece of software that is not AI and says: if the AI model recognizes the pedestrian, we do not run over the pedestrian; if the AI model recognizes a red traffic light, we stop.

The thing about statistical engines is that they are based on probabilities, not on deterministic logic. Systems make errors all the time, and it is impossible — technically impossible — to get them to stop making errors.

> Humans make errors way more than—

Systems in this case do, too. The safety record is, isn’t it, like 10 times more safe to be driven in a Tesla with autonomous driving than it is for a human to drive?

It depends on the place. It depends on whether the Tesla was trained to specifically navigate the place that you are driving.

> Get drunk—

Because if it is in Mumbai, or in some place in Vietnam, no, it would not be safer. I would much rather be driven by someone who has been driving in that place their whole life. I am not arguing against the fact that, in certain places where the car has been explicitly trained to drive in this place, it has a better safety record than the humans driving in that place. But you specifically asked if I think that all of the most cars in the world in the United States, since we are here, will be driving themselves.

I do not actually think that is imminently on the horizon.

> 10 years.

No, I do not think so.

I sat with Dara from Uber, and he is pretty convinced that his 9 million couriers will be replaced by autonomous vehicles.

How long have self-driving cars been invested in thus far? It has been more than 10 years. And what percentage of cars right now are autonomous on the US roads? Part of it is that it is actually not a technical problem. Part of it is also a social problem: do people even trust getting into these vehicles? Part of it is also a legal problem, which is if the self-driving car kills someone — which it has happened.

> Yeah, it has happened.

Who is responsible? In the case in LA, it was both Tesla and the driver because the driver dropped their phone, looked down, and this was a couple of years ago, I believe, and then went to grab their phone and hit someone. It went to court, and they were held both responsible, both the driver and Tesla.

In terms of Tesla, pretty much everyone that gets the car, it comes with autonomy now for pretty much most people, I believe.

> Partial autonomy. Yeah, it’s called full self-driving at the moment where it’s like—

I mean, yes, it is called full self-driving.

> Full self-driving supervised where you kind of have to be looking in the d. You have to be looking in the right direction, but—

Yeah. So, it is partial autonomy.

> And here in Austin, it’s full autonomy because there’s no steering wheel.

Yeah.

On the new car, so you cannot drive it anyway. But it is, you know, the Model Y is the undisputed highest-selling car, best-selling car in the world across all brands.

Well, my point here is that these predictions where they say AI is going to completely change transportation and driving, it is going to completely change lawyers, accountants are not going to have jobs — do you believe that they are true? Do you believe that there is going to be mass job displacement?

#### Employment impacts and layoffs

Okay, so I do think that there is going to be huge impacts on employment, and we are already seeing those impacts. It is not simply because the AI models are just automating those jobs away. It is specifically because the models are improving in certain capabilities based on what the companies that are developing them choose to improve them on. And executives at other companies are then deciding to fire or lay off their workers because they think that AI can replace the worker, irrespective of whether that might be true. And there,

### Job displacement and failed replacement plans

There have been cases, as with the Claer CEO, where he laid off a bunch of people thinking he would replace everyone with AI, and then it did not actually work, so he had to ask some people to come back.

> I actually DMed him about this. If you're hearing this, this is because I've DMed Sebastian and he's fine with me sharing this.

He said that he had heard Sebastian's name mentioned a lot, and when they talked about AI in the past and people mentioned Sebastian and Claer as the example, he wanted to clarify with him what the truth was.

He said, "It's great to hear from you. I think sometimes people struggle with two things can be true at the same time. I think it might be time to come back on your podcast. To your point, this is the media misinterpreting my tweet. We are doubling down on AI more than ever. Claer is shrinking with almost 100 employees per month due to AI. We used to be 7,400 at the peak. A year ago, 5,500. Now we're 3,300. And by the end of summer, so this was last year, we'll be 3,000 people. AI handles 70% of our customer service conversations at this moment. This is because we have realized that with AI, the production cost of software comes down to almost zero. Just like manufacturing used to be all handcrafted and then the machines came. Code used to be all handcrafted up until a few years ago. And now it is machine produced. And ultimately we pay people more than ever for the unique handcrafted man-made stuff. China is a bank. People will want to connect to humans not only machines. They want us to be personable, relatable, even flawed. So we need to make sure while we are automating replacing with AI in parallel, we make sure we offer a super available human experience."

She is really glad that this was read, because it touches on some really important nuances about AI and the impact it is going to have on employment. There are often these binary narratives: AI is going to come for every job, or people say AI is not actually working and it is not actually coming for jobs. The reality is that it is coming for jobs. There are definitely jobs that are being automated away because of the capabilities of the models. And there are also jobs that are being lost because executives are deciding to lay off the workers even if the models do not match the capabilities, because it is good enough. They would rather have the good enough model for way cheaper, or they made a mistake with hiring. They blew up their team, and it is a great convenient thing to say.

Exactly. There are many reasons, but clearly we are already seeing impacts on the job market. The U.S. jobs report that came out earlier this year showed that there has been a decline in hiring, a slowdown in hiring, especially across white-collar professional industries. And then Anthropic's report came out this week. The TLDR is that it matches what was being said: Anthropic looked at exactly how people were using their models, and they looked at what people are saying. They said that there has been a 40% reduction in entry-level jobs in particular, and then they made this graph that has gone viral over the internet. The red shows where we are now in terms of capability, and based on how people are currently using the models, they predicted, extrapolated out, that the blue part will be the disrupted parts. This is the things that they say AI can do right now, but people do not realize it yet.

If you look at it, it is kind of all the stuff you would expect: the physical real-world human stuff, which robots maybe can do someday, like construction or agriculture, are untouched, but office and admin, finance, math, law, media and arts. That is all the stuff just named that they purposely highlighted. Finance, math, law, media and arts. "That's me cooked." Office and admin. They do focus a lot on assistant-type and managerial work.

But the other thing that the Claer CEO said was that people also want human experiences, so it is not actually just about the capabilities of the models. It is also about what people want. Some things they would turn to AI for, and some things they would not, irrespective of whether or not AI is capable of doing it, but because of a preference for human-to-human interaction.

So what we are seeing right now is the thing that happens with every wave of automation: a bunch of entry-level work gets automated away, and there are also new jobs created. But the jobs that are created fall into one of two categories. There are people who get even higher-skilled jobs, and as he was saying, we pay people more for the handcrafted code now. And then there are the people who get way worse jobs.

There was an amazing article in *New York* magazine talking about how a lot of people get laid off and then end up working in data annotation, which is the labor that has been referred to throughout this conversation that companies need in order to teach their models the next thing the companies are trying to automate. So a marketer gets laid off and then goes and works for a data annotation firm to train the models on the very job they were just laid off from, which will then perpetuate more layoffs if that model develops that skill. The article talked about how this has become a huge catchall for a lot of people who are struggling to find job opportunities right now, including award-winning directors in Hollywood who are actually secretly doing this data annotation work to put food on the table.

So when people talk about mass unemployment and then say there will be some new jobs created that we cannot even imagine, a lot of these narratives rarely talk about, first of all, why some jobs are going away. It is not just because of model capabilities; it is also because of executive choices and because of the rhetoric they use if they want to downsize. But the other thing that is rarely talked about is that a lot of the jobs that are created are way worse than the jobs that were there.

It breaks the career ladder. It is the entry-level and the mid-tier jobs that get gouged out. There are higher-order jobs and then way more lower-order jobs that get created. So how do people continue to progress in their careers? There are no more rungs on the ladder.

I actually do not know the answer to this question, and I have been furiously trying to find a good answer because everything is theory. For my audience, I would say most of my audience do not run...

### Company hiring claims versus reality

Businesses. A lot of them do, a lot of them aspire to, but they do not run businesses. So they are kind of in the land of theory. They are hearing lots of different things. Jack Dorsey does his tweet saying he is halving his headcount because of AI. They do not know what is true. They do not know the internal economics at Jack’s company, and whether he bloated the company during the pandemic and is just using this as an excuse to make the share price spike seven points because his investors now think they are an AI company or whatever.

> **Mh.**

It is hard to pass through. So eventually I go, okay, what am I doing?

> I have hundreds of team members, probably 70 companies I invest in, maybe five or six that I am the lead shareholder in. What am I actually doing on a day-to-day basis right now? I am also, I also consider myself to be head of recruitment.

But in the last month in particular, I have met extremely capable candidates in terms of cultural alignment, hard work, those kinds of things, but I have had to take a great deal of pause because when I run the experiment of can I get an AI agent to do that exact same thing, the answer is increasingly yes, especially in a world of open clause.

So what I am curious about is, now you confront this decision where you are seeing in this short-term period you could just choose the AI agent and in the long-term period there is no career ladder. So who are you promoting into these senior roles? How do you resolve it for your own company?

> **Yeah, it is a good question.**

So there are kind of two ways I am thinking about it. I think really deep expertise is very, very valuable because if you are now the orchestrator of potentially AI agents, it is really about having a deep understanding of the right question to ask, and that is someone who has deep expertise on something. So I need my CFO, because if she is going to be orchestrating our team of agents that might be doing financial analysis or whatever else, she needs to understand what to tell them to do in our company.

> **Mhm.**

And in turn, financial analysts cannot do that. They need the 50-odd years of experience that CLA has. On the other end, I need Cass. Cass is 25. Cass knows everything about AI agents. He is a young Japanese kid who is highly, highly curious. On the weekend, he is building AI agents to solve problems in my life. I need those two kinds of thinking, which is highly proficient agent-maxing young kids, or they do not necessarily need to be young, but really lean in with high curiosity. That is creating a force multiplier in my business. And then I need deep expertise.

Now the everything else outside of there is another group I have thought of: people with extremely great IRL people skills, because we do meet people in real life. We greet you when you arrive here. We greet when we go for lunch with big clients that we have, whether it is Apple or LinkedIn or whoever it might be. We need to smoosh.

> **Mhm.**

And we have teams who are in person in the office. So we do a lot of stuff IRL, and increasingly we are building communities even for this show. We are doing community events all around the world. So we need people who are good at that as well: IRL, bringing people together in real life, and organizing stuff. Those are the three groups of people that I think are irreplaceable right now.

And if you were to take all of the roles that could be done by AI agents, and replace them with AI agents, do you think you would still have these three pools of people to hire and promote into the three critical things that you need in the long term?

> **If things carry on at the current rate of trajectory,**

yeah,

> **one could assert that even those roles would experience pressure. If you just imagine people think of things either statically or linearly or exponentially.**

Yeah,

> **you imagine an exponential rate of improvement, which is kind of what I have seen. Even like a 10% compounding rate of improvement at some point,**

at some point, at some point, I think

### What AI cannot replace

What remains is actually the **IRL, irreplaceably human stuff**: human to human, our Maslovian needs of being in person like we are now, and that is not going to change. We need connection. Humans get very sick when they do not have other human beings in their life and strong, deep relationships. **100% agree.** That stuff is going to matter a whole lot.

I have this contrarian, weird take that maybe this is the first technology that is going to deliver on the promise of making us human and connected, because we are going to be rendered useless at everything else other than what humans are good at. Because all the other technology said, *“Oh, we’re going to make you more connected, connecting the world,”* and they disconnected the world and isolated the world. But maybe this is the one. It is so intelligent now that it does not need us around in spreadsheets anymore.

> Do you see that actually happening in real time right now, that it is making us more able to be in person, connected with one another, having deeper social community engagements?

**Yes.**

**Yes.** And I will give you some data points.

#### Social media plateau and dark social

Data point number one: the *Financial Times* released a report on social media usage, and what they saw is that **2022 was the peak and it has plateaued ever since**. The generation that has plateaued the fastest and is heading down is the younger generations. The boomers are still off to the races, right, on Facebook and stuff.

Then you look at the way **Gen Alpha** are using social media. They are not posting as much. They call it **posting zero**. They are scrolling sometimes, but they are in **dark social environments** like WhatsApp, Snapchat, and iMessage. They are not performing to the world. They also value **IRL experiences** much more than any other generation. They are not getting smashed. We are seeing every brand has a **run club**. Runs are exploding around the world, and we are seeing this real sort of, almost like innate, realization that technology let us down at some fundamental level. Dating apps let us down. Social networking kind of has let us down. And we are seeing, I think, maybe a **bifurcation of society**, where a lot of people are going, *“I want to go back to what it is to be a human.”*

I would imagine that in such a world, where intelligence is so sophisticated that we no longer needed to sit at laptops, screen time is going to continue to fall. You go into an office and you are not going to see people sat at laptops. You are going to see something completely different.

#### Robots and the next workplace

And then we talk about robots and **Optimus robots**. Elon says there will be **10 billion Optimus robots**. Elon has been wrong with timing before. He is almost never been wrong on the big things completely. He just has a bad track record on timing. Um, so I...

### Robots, agents, and changing work

#### Robots, factory work, and human value

Think he is probably right. I have got some people on the way from Boston Dynamics and these other big companies like Scale AI, and they are actually bringing the robots here to show it, like folding laundry, doing the dishes. I am not saying that is what I would want in my home, but I think factory work is going to completely change. I think a lot of manual labor is going to completely change, and I think we are going to be forced to do what only we can do.

#### Checking in with Sebastian, CEO of Cler

Sebastian, who is the CEO of Cler, has actually just called me.

> Hello, Sebastian.

> You are right.

> Hey, how are you?

> I am good. How are you? It has been a while.

> It has been a while since you were on the show. I was just saying we do need to get you back on.

He said he just had a couple of simple questions, because he does a lot of interviews, and Cler has always been mentioned. He thought the media had said that you double down on AI and then you reversed because it did not work out. He knew he had spoken to you a while ago and exchanged a couple of DMs about it, but that was more than a year ago now. So he wanted an update on Cler’s business, AI agents, and all of that, if possible.

First and foremost, Cler was early on released AI to support customer service, which had the initial benefit of more calls being dealt with by AI. Customers liked that because those calls or chat messages were much, much faster and more qualitative. Since then, that has actually expanded slightly. What they did try to communicate as well was that they believed in a world where AI is cheap and available, the value of human interaction will be regarded as higher. So the future of customer service VIP is a human. They have therefore doubled down on providing more of that, but at the same time the efficiency gains within the company have continued.

I mean, they used to be about 6,000 people, and now they are less than 3,000, which is two, three years since they stopped recruiting. At the same point in time, their revenue has doubled. So you can clearly see that AI has allowed them to do more with fewer people, but they have avoided layoffs and instead relied on natural attrition when people move on to other jobs. From his perspective, they will continue to be very not really recruiting much. They recruit a little bit here and there, but they expect that natural attrition of 10 to 15 percent per year to continue and to become fewer.

He thinks the big breakthrough was really in November and December last year, where even the more skeptical...

### Coding’s shifting future

Engineers who were very well renowned and appreciated, including the founder of Linux and others, basically said that coding has now been resolved and hence, you do not need to code anymore. That was kind of a common sentiment. So I think in coding, that is definitely an engineering work that has seen a tremendous shift in the last six months.

> What do all these people go do, Sebastian?

> I am optimistic. I mean, I think obviously people will have a lot of opinions about this topic, but I still believe that we are going to move towards a richer society. Now, in the short term, there could be more worry about what happens if people do not get a job and so forth. But I think in the longer term, I am optimistic what it means for society and humanity.

> Thank you so much, Seb. I will chat to you soon. Thank you for taking the time. I appreciate you, mate. Thanks.

> All right. All right. Bye-bye. Bye-bye.

You know the little traditional SIM card that goes inside our phones? They have not changed at all since they were invented in the '90s. You have this physical piece of plastic that means you are locked into one carrier, one network, and the second you cross a border, that carrier can start charging you whatever they want.

But there are alternatives, and today's sponsor, SY, is one of them. It is an eSIM app that gives you a safe and secure data connection in over 200 destinations. All of their eSIMs have built-in cyber security, which is great if you are traveling for work and looking at confidential material.

I have been using SY whenever I travel because the connection is always reliable and it saves me a ton of roaming fees. It also means I do not have to deal with all of the faff that surrounds sorting out a SIM everywhere I go. If you want to give it a try, download the sale app from the app store now and scan the QR code on screen. And if you want 15% off your first purchase, use my code `DOA` when you get to checkout. That is `DOA` for 15% off. Keep that to yourself.

This is something that I have made for you. I have realized that the Diary of a CEO audience are strivers that we want to accomplish. And one of the things I have learned is that when you aim at the big, big, big goal, it can feel incredibly psychologically uncomfortable because it is kind of like being stood at the foot of Mount Everest and looking upwards. The way to accomplish your goals is by breaking them down into tiny small steps. And we call this in our team the 1%. And actually, this philosophy is highly responsible for much of our success here. So, what we have done so that you at home can accomplish any big goal that you have is we have made...

### Data labeling and working-class disruption

#### The New York Magazine piece and the working class

What you are describing is true for a very specific category of people, which is often the business owners and leadership within companies that can actually make these decisions about how they spend their time and what they ultimately do with their time. But the piece talks about the working class: people who are not business owners, who are then having to experience being laid off and then working for the data annotation industry, which is now one of the top jobs on LinkedIn, by the way.

LinkedIn had a report that showed the top 10 jobs with the highest growth in the last year, and data annotation is on that list.

#### What data annotation is

For anyone who does not know what data annotation is: it is the process of teaching these chatbots, or any AI system, to do what they ultimately are able to do. The fact that ChatGPT can chat is because there were tens of thousands or hundreds of thousands of people who were literally typing into a large language model and showing it, *this is how you are supposed to then respond when a user types in a prompt like this*.

Before they did that work, ChatGPT did not exist. It would just generate some text that was not in dialogue with the person; it would generate something that was adjacently related.

Is this what they call reinforcement learning, where you give it a —

It is a part of the process of reinforcement learning. You do data annotation, which is literally showing lots of different examples of things that you want the model to know, and then reinforcement learning is getting the model to then train on those examples iteratively in a way that gives the model some of those capabilities.

#### The people being absorbed into the industry

What the New York Magazine piece highlighted is that many of the people getting laid off now, or struggling to find work, are highly educated people: college graduates, PhD graduates, law degree graduates, doctors, and, again, award-winning directors who are then struggling to find employment in the economy because the economy has been very much restructured by AI.

They are then finding themselves serving this industry, and the industry is designed in a way that is extremely inhumane.

The companies that use these data annotation services — these third-party providers, these data annotation firms — OpenAI, Grok, Google: they will hire these firms to then find the workers to perform the data annotation tasks that they need for these systems. These firms, these third-party firms, are incentivized to pit workers against each other because they want this data annotation to happen at speed and as cheaply as possible, so that they can also compete with one another in this middle layer to get the bid, the contract, from the client.

And so all of these workers who were interviewed for this New York Magazine story talk about how they actually no longer have an ability to be human because they are waiting at their laptop to be pinged on Slack for when a project is going to open up for data annotation. They have tried job hunting. They literally cannot find anything else. This is the thing that is going to help them put food on the table for their kids.

There was this one woman who said, *I have so much anxiety about when the project is going to come, when it is going to leave, that when the project came, it was right when my kid was coming off of school. And I just started tasking furiously because I do not know what is going to go and I need to earn as much money as possible in this window of opportunity. So then when my kid came home and tried to talk to me, I screamed at my child for distracting me.*

And then she was like, *I have become a monster and I am not even allowed to go to the bathroom or take care of my kids, let alone myself, because this industry that is absorbing more and more of the workers that are being laid off is mechanizing my life, atomizing my work, devaluing my expertise, and then harvesting it for the perpetuation of this machine that all of these AI executives are saying is then going to come for everyone else’s jobs.*

So what you were saying about this class of workers, the business owners that get to become more human because there are all of these AI models now doing the tasks that they do not have to do anymore: it is at the cost of the vast majority of people who are not business owners, who are struggling to find work, getting absorbed into the work of providing these technologies that the business owners can use.

Instead of becoming more human, they feel like their humanity has been squeezed and diminished, and they have no ability to have control, agency, and dignity in their lives anymore.

#### The scale of retraining

I think this is a big question that pertains to this graph here, which is that all of these people — if we believe Anthropic’s prediction of who will be disrupted — these people in industries like arts and media, legal, life and social sciences, architecture and engineering, computer and maths, business and finance and management, and also office and admin, would have to retrain at something else.

And unlike the Industrial Revolution, where you might get 10, 20 years to retrain because factories take a long time to build, the distribution layer that AI sits on top of is the open internet. So this is why ChatGPT can go and get hundreds of millions of users in no time at all and become the fastest

### Speed, inequality, and social fallout

The concern is that this disruption takes place at a speed where we cannot transition. That sentence was in the passive voice, but the real question is: **who is driving that speed?**

It is the companies, and their race with one another.

That means they are driving the transition to happen at a speed at which it would be really hard to take care of all of the people who would be bulldozed over by it. This is one of the crazy questions that no one can answer when faced with these AI CEOs: *What happens to the people if this is going to happen at super speed?*

The comparison was made to what the Uber CEO, Dara, said: there will be data-labeling jobs, for example, for the drivers. But they cannot all become data labelers. And there is a question around meaning and purpose and fulfillment. That comes from losing your meaning in life.

There are also the human consequences that show up in real families and real lives. So many people talk about how their father lost his job in Iran, or in some other country, and came to the United States and had to become a toilet cleaner. In one particular case, the person had been a doctor in Iran but came to the U.S. and was a toilet cleaner. He had to deal with the shame that he felt, the lack of dignity that caused, and how it affected his self-esteem. Depression and alcoholism then transpired from that.

If this happens at a large scale across society, there are going to be a ton of consequences like that.

This is the core of the work. The criticism of these companies is that they are creating technologies in a way that creates the haves and have-nots in an extreme form that already exists. It is exacerbating the inequality already seen in the world. The people who have things will have way more riches. They will have way more free time. They will be allowed to be more human. But the people who do not have things are being squeezed even more.

And it is not just from a work perspective. There is also an environmental and public health crisis that these companies have created by building these colossal supercomput facilities in communities all around the world, and they specifically pick some of the most vulnerable communities.

We are sitting in Texas right now, and OpenAI’s largest — one of its largest — data center projects is being built in Abilene, Texas, as part of the Stargate initiative, which was an ...

### Data centers and local community costs

This facility, announced at the beginning of Trump’s second administration as part of an effort to spend **$500 billion** on AI computing infrastructure, is projected to consume more than a **gigawatt of power** when it is finished — more than **20%**. That is actually a little bit inaccurate now; this was something that circulated online for a while, but there are updated numbers.

> Just for someone that can’t see because they’re listening on Spotify or something: it’s a picture of the size of this facility.

This is not the Abene, Texas one. This is a Meta facility. So first, let’s talk about OpenAI’s facility in Texas. That one would be the size of **Central Park**, and it would run **a million computer chips**. It would require the power of more than **20% of New York City**.

One of the things that is confusing is the idea that the job disruption promises were not real. That is not what was being said. What was being said is that when we talk about what these executives predict about the future, we need to understand that they are ultimately trying to influence the public in a way that allows them to continue maintaining control over the technology.

But objectively, do I think that the job disruption they talk about is real? There are already job reports showing that there is a restructuring of the economy happening right now. I do not want to comment specifically on that chart, but the restructuring is already happening.

Going back to the data center: this supercomputer facility, a Meta supercomputer facility, is being built in Louisiana. It would be **four times the size** of the Abene, Texas one and use **half of the average power demand of New York City**. It is the size of Manhattan. That makes it sound like almost all of Manhattan, but it would actually be **one-fifth the size of Manhattan**.

When these facilities go into communities, power utility increases and grid reliability decreases. The facilities also need fresh water to generate the power for powering them, as well as fresh water to cool them. There have been many documented stories of communities that are already severely constrained in their freshwater resources. They are under a drought when a facility comes in, and then people in the community are literally competing with the facility for fresh water. One of those communities is discussed in the book.

Sometimes these facilities do not connect to the grid. Instead, a power plant pops up next to them. In Memphis, Tennessee, where Musk built **Colossus**, the supercomputer for training Grok, he used **35 methane gas turbines** to power the facility. This is a working-class community, a Black and brown community, a rural community that was not even told it would be hosting this facility. They discovered it because they literally smelled what seemed like a gas leak in all of their living rooms. That is when they discovered that the methane gas turbines were taking away their right to clean air.

This community already had a history of environmental racism. They had already struggled a lot to access their right to clean air. And now there is this huge supercomputer in their midst, pumping **thousands of tons of toxins** into their air, exacerbating the asthmatic symptoms of children and the respiratory illnesses of other people. It is one of the communities with the highest rates of lung cancer.

And that supercomputer is taking their jobs too.

That is what is meant by the halves and the have-nots being pulled apart even further. If, in this version of Silicon Valley’s future, you are in the unfortunate category of being a have-not, then we are talking about you getting a job that is way worse than what you had before, because you might be doing data annotation and being treated as a machine rather than as a human, in order to extract the value of your labor for perpetuating this labor-automating machine that these people are building. You might be competing with these facilities for freshwater resources. They are also polluting your air. Your bills have

### AI as a transportation analogy

increased. So the affordability crisis is getting worse. How is that making people able to be more human?

> **What do we do about it?**
>
> **Yes.**

One of the analogies that I always use is that AI is like the word transportation. Transportation can literally refer to everything from a bicycle to a rocket. And we have nuanced conversations about transportation where we always say we need to transition our transportation toward more sustainable options. We need a transition toward public transport, electric vehicles. We do not ever say everyone should get a rocket to do every one of their transportation needs, right? We are in Austin. If you use a rocket to fly from Dallas to Austin, that would just make no sense. It is a disproportionate use of resources to get the benefit of getting from point A to point B.

This is how we should think about AI. All of the models that we have been talking about, I like to think of them as the rockets of AI. They use an extraordinary amount of resources and they provide benefit, some dramatic benefit to some people, but they are also exacting an extraordinary cost on a large swath of people because of the costs of developing this technology.

Why do we not build more bicycles of AI? Things like DeepMind's AlphaFold, which is a system that predicts how proteins will fold based on amino acid sequences. It is really important for accelerating drug discovery and for understanding human disease, and it won the Nobel Prize in Chemistry in 2024. And the reason why it is a bicycle of AI is because you are using small, curated data sets. You just have data that has amino acid sequences and protein folding. That means you need significantly less computational resources to develop the system, which means significantly less energy, which means less emissions, so on and so forth. And you are providing enormous benefit to people.

> **It feels like the horse has left the stable in this regard because they have already taken people's IP, they have taken media, they train on this podcast. We know they do because it shows that they do. I think there is a button actually in the back end of YouTube now that allows you just to click it and it says we will train on your YouTube channel. So the horse has kind of left.**

Here is the thing. If the horse truly had left the stable, they would not have to train on anything anymore. Why is it that their appetite for data has actually expanded? It is because, in order to build the next generations of their technologies, in order to have the technologies continue to be relevant and continue to update with the pace of new knowledge creation and society's evolvement, they need to train again and again and again and again. And why are they employing actually more and more and more data annotation workers over

### Public resistance and alternative paths

It is because they need more and more of that work over time. I have been reporting on data annotation work for over 7 years now, and it has not gone down. It has increased.

> **Do you think there is any chance of it going down?** Do you think there is any chance of this sort of brute force scaling approach, where you take data, you take computational power, energy, and you, you know, you have the data labelers and, you know, building out more and more parameters for the models? Do you think there is any chance it is going to stop or go in a different direction other than the one it is going in now?

> **I would love to reframe the question and say what should we be doing in this moment** where it is not going down, where we do recognize that actually these companies in this moment need continued resources, inputs and labor to perpetuate what they are doing.

> **Yeah.** Because this sounds like stop.

And I just feel like stop is like a HUD. It feels like I just think, you know, with the government in place they are supporting these companies like crazy. Globally this is happening. So I am like stop does not feel...

I always say we need to break up the empire and we need to develop alternatives, and we are already seeing a flourishing of incredible grassroots movements that are applying an enormous amount of pressure to the way that the empire is trying to unfold its agenda. **80% of Americans** in the most recent poll think that the AI industry needs to be regulated.

> **Yeah.**

When was the last time that 80% of Americans were on the same side of an issue?

> **No. Yeah.** When I have these conversations on the podcast, the comment section is clear.

> **Yeah.**

There is no disagreement. There is no one in there going, "Oh, no. I think they should crack on."

> **Yeah.** Dozens and dozens of protests against data centers have broken out all around this country and all around the world.

So, what do we do about it?

These are people that are doing something about it. They are actually reasserting their agency and exercising democratic contestation against the ways that the empires are going about their business.

What goal should we be aiming at? So, if I said to my audience, Janet at home, because this is kind of what I see in the comments, it is hopelessness. It is like, what can I do? I am just a...

> **Yeah.** Well, well, well, the goal is not that we completely get rid of this technology. The goal is that these companies need to stop being empires. And the way I define a typical business versus an empire is that the empires are predicated on this idea that they do not have to provide a fair exchange of value with the workers who work for them or the people who use them or all of the other people that are involved in the supply chain of producing and deploying these technologies. They can extract and exploit and extract and exploit and get more value than what they offer. Whereas typical businesses, there is a fair exchange. You buy a service, you feel like you got the same amount of value as the service that you provided. But for these data annotation workers, for example, they do not feel in any way that they are being paid the same value that they provide to these companies. So that is, for me, the north star: we should be pushing back and holding accountable these companies when they operate in an imperial way.

And that is what we have seen with all of these people that are now literally protesting in the streets against data centers and having an enormous effect, by the way, actually stalling data center projects and also completely banning data centers from being developed in their localities. We are seeing that with artisan writers that are suing these companies for intellectual property infringement and creating a huge public conversation about what it is that we actually, how do we actually want to protect our intellectual property?

I met Megan Garcia three weeks ago, who is the mother of **Sewell Setzer III**, the 14-year-old who died by suicide after being sexually groomed by a character chatbot. She was incredibly devastated by what had happened to her son. She also decided to do something about it. She sued the companies, and that lawsuit then sparked many other parents and families who were actually experiencing similar things to sue these companies as well. That has created an enormous public conversation about what these companies are actually doing when they exploit and they extract. What is the cost to the lives of people around the world, including children?

> **So, what do you think my audience should do if they agree with everything written in your book, *Empire of AI*, *Dreams and Nightmares*, and Sam Mortman’s *Open AI*?** If they agree with everything said here, if they agree with everything we have discussed today, they are concerned about their kids, they do not want everyone to become data labelers, they do not think that is a particularly great solution, what can they actually go and do?

> **When I was writing the book, the only discourse that was happening was this is the best thing since sliced bread.**

> **Mhm.** Because of all of the actions of these people, like saying when they are not happy with the things that these companies are doing, we now have 80% of Americans that want to regulate this industry. And so I would say to people, think about all of the ways that your life intersects with the resources and the things that the AI industry needs to perpetuate what they do, and also the spaces that they would need to deploy these technologies to continue having broad-based adoption in their work.

So you are a data donor to these companies. You could withhold that data, and that is what those artists and writers are doing. They are suing these companies to withhold, to try and create mechanisms by which that data would then be withheld.

You probably have a data center popping up around you. If you are at a school environment or a company environment, you are probably having a discussion in those environments right now about what should the AI adoption policy be?

And these companies, I was talking with some OpenAI employees just the other day, and they were telling me that it is understood internally that the revenue targets for the company are extraordinary and they need things to go flawlessly for it to all work out. And so they would need every single person to adopt this, every single space to adopt this. They would need to be able to build their data centers at the speed that they are trying to build them.

So what I would say to everyone of your viewers is: let us not make it go flawlessly if we do not agree with what they are doing.

> **Ah, okay. I got you.**

And then let us build alternatives. Because the thing is, what I am saying is not that these technologies do not have utility. It is that specifically the political economy that has emerged to support the production of these technologies right now is exacting a lot of harm on people. But

### Benefits, costs, and smaller AI models

We have research that shows that the very same capabilities could be developed with much more efficient methods, with much less resource consumption. And we have a lot of different other AI systems at our disposal that are like the bicycles of AI, that we also know provide extraordinary benefit at very little cost. So let’s break up the empire and let’s forge new paths of AI development that are broadly beneficial to everyone.

> It’s strange. I think I’ve trained myself to deal with dichotomies in my head. And this, for me, is such a dichotomy where I, as a CEO and as a founder, as an entrepreneur and someone that loves technology, think it’s incredible. It’s absolutely incredible, AI. It’s just so amazing and incredible, the things it’s enabled me to do and create.

> Yeah. Because it’s designed to enable people like you.

> And my car driving in the morning and being safer. Incredible. I think the billion-odd people that use AI tools or chat or whatever it might be, they’d probably say that it’s added value to their life. But, and this is the part that people find confusing, you can—and I invest in companies that are heavily using AI—but is it possible to think that is true and also think that there are significant unintended consequences, which the history of technology should have taught us to take a moment to pause and talk about because...

> I think this is absolutely like you can have both of these things in your head, and what I’m saying is that this tension doesn’t have to be a tension because we could actually preserve the utility and benefits of these technologies, but actually develop and design them in a different way that doesn’t have all of these unintended consequences.

> Yes. And I think there needs to be a big social conversation, which is why I have so many conversations about AI on the show. There needs to be a big social conversation about being intentional about the social impact, the social and environmental impact, and that conversation is not being had in government. From what I can see, the conversation takes place in the industry, and actually trying to pull it out of the industry and open people’s minds to it is hopefully what we’ve been doing over the last couple of months with this subject because...

> I think it’s actually been happening everywhere outside of the industry, and for local governments and state-level governments there have been huge conversations about this everywhere. Like I’ve been on book tour, I’ve been to dozens of cities around the world. People are having these crucial conversations everywhere. I have not gone to a single city.

> Yes. Everywhere. Even here in South by.

> Yeah. I haven’t gone to a single city.

### Closing reflections and advice

where the room is not packed and people are not wrestling with the same exact questions as every other person in every other room that I've been in.

> **Speaking of packed rooms, I know you've got to go because you've got to talk today. So I'm going to—we've got a last question, which is the closing tradition on this podcast.**
>
> **How would your advice to a friend with a terminal diagnosis differ from what you would do yourself?**

> **That's a great question.**
>
> **Differ from what you would do yourself?**

> **Oh my god. I have—I would tell them, like, enjoy, like, live life for yourself. You wouldn't do it and take it easy. And yeah, I am not taking it easy.**

> **Well, I think it's a good thing you're not taking it easy because you're leading a conversation which is incredibly important. And I think that's the thing. I think the conversation is the important thing. And so, you know, because of algorithms and echo chambers, it's so rare to have a conversation these days, especially a long form one.**

> **I agree.**

> **Like this. So, I think they're so important. And your book is for anyone that's curious about—**

I think a lot of people would have learned a lot of stuff today because I sit here and interview AI people all the time, and I've learned so much today. From reading your book and the extensive, objective perspective that your book takes, you're able to unravel all of these stories that we sometimes see in tweets and we don't know if they're true or not because you've gone and met the people and you've done your research, and you're an incredibly intelligent person, an extremely intelligent person who clearly has humanity's interests as your north star, and that shows up in everything you do and everything you say. So please continue to fight in the way that you are, because it's an incredibly important one. People like you are, I think, galvanizing the world to take the collective action that we're starting to see everywhere.

> **Yeah.**

*Empire of AI: Dreams and Nightmares in Sam Alman's Open AI* by Karen How. I'll link it below for anyone that wants to read this book. I highly recommend you do. It's a New York Times bestseller for good reason. Karen, thank you.

> **Thank you so much, Stephen.**

YouTube have this new crazy algorithm where they know exactly what video you would like to watch next based on AI and all of your viewing behavior. And the algorithm says that this video is the perfect video for you. It's different for everybody looking right now.