So there is this plane taking off, and we’re clinging on, and it sounds like you also have a pretty similar mental model. Yeah, we’re clinging on, but you have this optimism that the clinging is probably going to work, correct?

You know, stepping back, just in terms of basic mental model here, I use the metaphor of climbing a ladder really fast. Another metaphor is like, “Hey, there’s a plane taking off.” I’m imagining this of the human race clinging to the plane that’s taking off, right? Because the AI ...

Okay, so where you and I disagree is you just have this huge optimism of, “Oh, yeah, it’s… we’re gonna climb this huge scale. It’s going to go way beyond anything that we can really imagine today, but the humans are just gonna be climbing the ladder. We’re never going ...

It’s the genius of the scenius, the scenius, right? Yeah, okay, I’ve heard that term thrown around before.

Okay, so I think I get what you’re saying. You’re saying that you actually agree with me that the intelligence spectrum does actually go much higher than humans today. So even if you look at a company of humans today, right, you look at all the coordination of a SpaceX, ...

Don’t you think the future is going to be a place where block-sized data centers or block-sized mounds of computation… there’s just going to be this entity that is just going to be more powerful than any human who has ever lived up to today, like all the humans together, ...

So you’ve got this city block of computation. Don’t you think that within a few years or decades, a city block of computation will surpass the capabilities of all humans in aggregate who are alive in 2025?

Right, it could. And would you also agree that you can have a mind, let’s say it just runs, it takes over a city block. It’s a city block full of computation, densely packed transistors, thanks to TSMC or whatever, right?

Let me unpack this, okay? So I’ll ask you a very fine-grained question here, which is just in the space of possible minds, doesn’t there exist a mind which is vastly superior in capabilities to the current human mind?

Okay, so then my question to you is, isn’t there an even more superintelligent superintelligence coming? Do you really think we’re hitting against the ceiling here?

Okay. Well, I guess I should have known you would say that, because you did write verbatim, you said, “The superintelligence we need is already here. It is the untapped potential of human collaboration. It is we, the people.” So you stand by that statement?

So to start that part of the conversation, let me just ask you, what’s your AGI timeline, very roughly?

But for now, let’s get into the, what I think is the crux, from what I gathered so far, the crux of where you and I disagree. If I’m guessing right, I think it’s our expectations of the emergence of vastly superhuman intelligence.

Okay, fair enough. So I won’t address that directly here, but for viewers of the show, you guys can go look up, there’s a whole other episode I did with Vitalik Buterin here on the show, and there’s also an episode I did where it was just me reacting to ...

Is there any particular point of disagreement between the organic alignment people and what you think about AI, or is it really strong agreement?

So let me just ask you one more question: Is this similar to Emmett Shear’s company, Softmax? What do you have in common with them?

You know, under the assumption that AI is not going to suddenly overwhelm us with its uncontrollable power, everything you’re saying is great. Let me just ask one question to finish up the section on kind of your worldview, right? We talked about plurality, and the 6-Pack of Care, and ...

Okay, so I’m getting the sense that where you and I are going to disagree is just in our prediction of what it looks like for AI to go superintelligent, and we haven’t touched on that yet. So I’m not directly disagreeing with anything you’re saying. I think you’re doing ...

So my question is, don’t you think that there’s this huge question mark as to whether this ideal is feasible at all?

It’s the same thing that I told Vitalik when he came on the show, and we were talking about d/acc. In the context of d/acc, I was saying, “Look, if somebody’s sitting on an island like Taiwan, and nuclear war is happening, or struggle for control over the Earth among ...

So I personally have a similar reaction to both proposals, which is great, I don’t object in principle, right? It’s nice if you can get it, right? So if we can head into the future, and we can always, like you say, actually listen to people, actually keep promises, yes, ...

Right, defensive. Yes, that’s right. Yeah, and I think you agree with all those Ds, and you’re adding a few more things that are maybe getting a little bit more in the weeds of the processes. Like, we check the promises, we check the results. That’s kind of new to ...

Right. Yeah, d/acc, I know the biggest ones are democracy, decentralization. I think those are the two biggest Ds. Maybe there’s another D worth mentioning.

Everything you’re saying about the 6-Pack of Care, the approach, has a lot in common, I’m sure you know, with Vitalik Buterin’s d/acc, right?

And sure enough, there’s this image of a six-pack, and this is what it says: “Actually, listen to people,” that’s one of the things in the pack, and then, “Actually, keep promises. We check the process. We check the results. As win-win as possible, and as local as possible.” All ...

Here, let me put it up on the screen. Here I am at 6pack.care. It’s a website that anybody can visit. It says, “6-Pack of Care, Institute for Ethics and AI, a research project by Audrey Tang and Caroline Green.”

Oh, there you go. All right, so it’s got two different senses. Okay, do you wanna go through the parts, or do you want me to read off what I have?

And by six-pack, do you mean like a six-pack of beer or six-pack abs?

Let’s also talk about a recent article, The 6-Pack of Care, right? That’s one of your key metaphors.

So in a bit, I wanna go a little bit more forward-looking and talk about the intelligence spectrum. But in terms of the optimal sequence of hitting on all the interesting points, let’s go back to some of what you’ve written.

Exactly, and it’s really aligned with you. And so the thing for me is that’s all great, it just doesn’t really get to the core issue for me, which is doom or human extinction from superintelligence.

And if I could personalize my feed, I’d be like, okay, predict how likely I am to be outraged and just give me two or three outrages a day. So I don’t need to be repeatedly outraged.

Yeah. Yeah, yeah, totally, totally, and I’m with you on this one because I am a heavy user of X, formerly known as Twitter, and a lot of the times I find myself getting outraged by positions that I strongly disagree with, and I can’t believe so many people are ...

Great. Okay, so yeah, personally, I can’t say I disagree. I mean, I think people should have freedom and flexibility. I mean, there might be some people who like it when they open TikTok, and they get really engaged for an hour, and they don’t have a problem shutting it ...

Yeah, okay, so that’s a very concrete example we can do. So if I understand correctly, you’re basically saying somebody who’s scrolling Facebook or Instagram or TikTok today, the maker of those products is going against plurality principles because they’re so focused on this metric of engagement, and so you ...

Okay, I mean, it sounds good. It’s hard to argue with, right? Because you’re evoking things that a lot of us like. We all want everybody to win, and we all want more connections among one another. So maybe the way to engage with this position best is to contrast ...

Yes. All right. Fair enough. So let’s go to plurality, because this seems to be a big unifying concept in all of your forward-looking proposals on how to think about AI, and you’ve already made a bunch of the points that I think are part of this, but maybe step ...

Yeah, hopemonger.

And so I consider this show… I actively consider myself a fearmonger because, you know, the same way they say, “Hey, you’re not paranoid if they’re really out to get you,” right? So in this case, I’m saying, you’re not panicking if you’re really about to all get… right, that’s ...

And so I actually see… I see us so far under the optimal level of panic, to be honest. Some panic is good, or some, you know, whatever you wanna call it, some attention, some fever, right? Raising the temperature.

It’s like, from my perspective, it’s like the whirling razor blades are just coming. They’re so close, and everybody’s not even opening their mouths and screaming yet.

Okay. I mean, I respect your perspective. My perspective is that everybody is still way too low on the doom, doomerism spectrum.

Okay, so if I understand correctly, there’s some optimal level, right, of the single parameter of how doomy people are. There’s some optimal level, and if you get beyond that level, it’s counterproductive. But do you think that we’re below or above that level right now?

Okay. So to me, this seems a little too conveniently clever, because couldn’t you also argue that if a bunch of people focus on AI doom, that’ll just make people fight to stop it and then lower the probability, right? It’s hard to say if it’s positive feedback or negative ...

Yeah, okay, okay. And we’re definitely about to talk about that more. I wanted to ask you, you’ve referred to AI doomerism as a hyperstition, right? Explain that.

Yeah, okay. I see where you’re coming from. Yeah, I mean, I agree, there’s positive side effects. One positive side effect is also even just if there’s a… even if hypothetically, there’s a low doom risk, it’s still nice to have as much control as possible over the AI you’re ...

Yeah, do you have any thoughts on the value of that statement or who else you would have liked to see sign it?

Okay. Well, that’s great. I think that’s productive that you signed that statement. I think a surprising amount of people signed it, right? Sam Altman, Dario Amodei, Demis Hassabis, all the top AI company leaders except Yann LeCun and Mark Zuckerberg signed it, so I was happy to see that.

So even today, heading into 2026, you still think that the risk of extinction from AI should be a global priority alongside pandemics, nuclear war, and other societal scale? You stand by that statement, correct?

The statement says, “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks, such as pandemics and nuclear war.” How did they approach you, and why were you so eager to sign?