Before we get to today’s guest, I want to start with a little story. In 2014, Taiwan’s governing party was trying to strike a trade deal with China. They wanted to do it in a way that was underhanded, they didn’t want the general public to have a say in how things went down. So, in response, thousands of protesters flooded the streets and they took over the legislature. For weeks, the world watched as young people in Taiwan fought for transparency and democracy, and they won. One of those people was Audrey Tang. At the time, she was a cyber hacker, someone who grew up on the internet who used her knowledge to spread awareness of the cause to supporters around the world. Within a year, however, Audrey had switched to the other team, she had joined Taiwan’s government and she was tasked with helping it become more transparent. Later, as digital minister, she became the point person for responding to cyber threats from China.

In her time in the public eye, Audrey has become an advocate for digital democracy as well as for a better, more moral AI. Audrey thinks we can do it with the help of technology.

I wanted to ask her about that. In a time when there’s so much distrust in tech, how can tech actually bring us together? I’ve had the pleasure of speaking with Audrey a few times before and, every time, I walk away in awe of how she thinks, she operates on a different-seeming, utopian, spiritual plane of existence. A friend of mine once told me that he thinks Audrey is the smartest good person on the planet. So, what does the smartest good person think about the future of AI and what it will do to the world? I’m delighted to share this conversation with you.

Audrey, it’s wonderful to see you again.

So good to reconnect after Paris.

We had a good time in Paris and we made a cool little video that, weirdly, a lot of people saw.

Yeah. On LinkedIn, it really went viral and I got a lot of inquiry. What do you mean by Buddha? What do you mean by egoless? What do you mean by horizontal?

I think maybe a good rule of thumb is that, if you make a viral video of two minutes in which you explain that you’ll get AI to summon a Buddha, you might get follow up. So, that’s what we’re going to do here today, Audrey. By the end of this podcast, everybody’s going to know what it means for AI to summon the Buddha. Deal?

Okay, let’s do that.

A lot of interviews with you start in 2014 when you come to power and you help lead a revolution in Taiwan but I want to start in 2022. And so, you’re there, you’re the digital minister, you’ve come to power through this people’s revolution, and you’re helping Taiwan’s democracy work. Nancy Pelosi, speaker of the House of the United States comes over to Taiwan, she leaves and then all hell breaks loose and there’s cyberattacks everywhere and this is partly your responsibility. So, take us back to that moment, tell us what happened, tell us what you did.

Certainly. So, we were already setting up the new Ministry of Digital Affairs, we even had a new website already online and were all very excited and then all hell broke loose exactly as you said. For example, hate messages featuring Speaker Pelosi outside of the signboards of the Taiwan Rail Station, even in convenience stores at 7-Eleven. And when people look at those hate messages thinking, “Oh, have the railroad been hacked in,” and so on, they got a lot of influence operations, rumor mills, propaganda on social media. And when journalists tried to check the ministry’s websites, well, they were very slow to respond because there’s denial-of-service attacks and a lot of different websites and so on from the schools to government ministries and so on simply become defaced or otherwise malfunctioning.

And so, yeah, this was the first time that we have seen the info attack and cyberattacks and other forms of gray zone attacks all coordinating together, probably attempting to sink our stock exchange. But unfortunately, they did not achieve the goal, the stock actually grew that day and I remember overcoming these rumors through humor. For example, saying to journalists, “How about let’s have a call to action? People who are downloading the Brave browser can help pinning our website on the decentralized interplanetary file system to keep us afloat despite the denial-of-service attacks.” And in doing so, people learned that keep dialing into a telephone line to keep it busy really isn’t the same as taking over the call center.

And of course, the signboards and so on which was manufactured by PRC brands in Beijing and Shanghai and these are swiftly disconnected from the internet and then we banned them, not just from the public sector as we always did, but also found extended networks that’s run by public sector places and so on.

That was an awe of society response exercise and it really strengthened the integrated nature of the ministry.

It’s an interesting situation. The Chinese government hacks, they have some effect, they create this evil Nancy Pelosi that’s taking over Taiwan and your response is humor to defeat rumor.

Had it been a more serious attack, had they actually taken down the electric grid, you couldn’t have used humor to defeat rumor, you would’ve had to have been very severe in your response. Tell me about how you calibrated the decision to use humor as opposed to, say, citizens come together and combat the enemy.

In humor, there’s also a real call to action. I’ll use another example. In early 2020, early in the pandemic, we started rationing out masks and, at the time, nobody quite precisely know how the mask interact with the new virus. There was people in Taiwan who said only N-95, the highest grade mask, is useful because of our SARS experience. Wearing a mask actually harms you and N-95 is the worst, and so, the science is out there, we don’t actually know. So, what we did, again, is to find this uncommon ground and we pushed out this public service announcement. A very cute dog, Shiba Inu, putting her paw to her mouth saying, “Hey, wear a mask to remind each other to keep your unwashed dirty hand from your face,” and we monitor tap water usage and the polarization just died down because people who laughed about the Shiba Inu message, reassociated mask use to hand washing which is a very non-polarizing topic.

And so, I do think that this humor over rumor is not to make this non-serious. Of course, a virus is very serious but we want to make it serious in a way that it can elicit what we call societal resilience. That’s to say people can respond in this uncommon ground despite their initial positional or ideological differences.

And this is part of the genius of what you’ve done which you’ve used, you and your colleagues, have used all these methods to somehow make people in Taiwan, as I can tell, trust each other more, trust the government more, and fight about stupid stuff less. One of the tools you’ve used to get there is humor over rumor. What are some of the other tools?

The other tool is uncommon ground defined as the few things that people didn’t even know they agree because of their ideological differences that can be somehow discovered through algorithm. And we now think algorithm and we think of the antisocial corner of social media, engagement through enragement, polarization, caricature of each other; but it doesn’t have to be this way.

In 2014, our approval rate of the president at the time from the citizenry was just 9% and we specifically designed systems at the end of 2014 to start recovering some of that trust by asking people very simple questions: How do you feel about this controversial policy? Maybe it’s about Uber, so: How do you feel about people with no driver license picking up strangers they met on an app on their way to work and charging them for it? How do you feel? And of course, people feel very strongly. Some people feel it’s extractive, some people feel it’s sharing, it’s carpooling, however, people, by and large, agreed in the middle that, for example, undercutting existing meters is bad.

So by making sure that people can post their feelings and see the reaction of each other’s feelings, something like the community notes of today but apply as a pro social media by the government at the time, we basically discovered the uncommon ground that unites the society despite differences and then the approval rate became more than 70% by 2020.

Explain the differences in how social media works in Taiwan versus the way social media works in the United States. We have Twitter, we have Bluesky, we have Threads, we have Instagram, we have all these big platforms controlled by large corporations. And on those corporations, people, when they talk about politics, they mostly fight, they don’t really find common ground. Explain the difference in Taiwan and the mechanics that got you to this different place.

So, in 2015, many social media companies switched from a feed that is the most recent post by the people you follow to the For You feed which means anything that keeps you addicted to your touchscreen basically.

Audrey, even as you say that, I feel like I want to check Twitter. I’m just kidding.

It’s a compulsion. I turned my touchscreen grayscale, I don’t feel that. Reality is always more vivid.

So, I assume also, while we’re at it, I’ve got to imagine you have no notifications on your phone. Is that correct?

That’s correct.

No notifications, grayscale, what else?

I mostly don’t interact through a touchscreen. I mostly interact through a stylus or keyboard or voice, anything that is intentional.

Intentional and a little more human, right? So, through the action, you’re reinforcing the humanity. But that’s you. And it’s not just that you’ve done a good job with social media, it’s that your whole country has done a good job.

So, in 2015, we figure out that we can flip this For You feed around. Instead of individualized recommendation—parasitic AI—we can have a For Us feed that fosters the kind of statements or feelings that connects people from different sides very concretely. When you get into the Uber discussion, for example, we use the Polis platform out of Seattle, and the platform basically shows you a fellow person’s sentiment and then you can say I agree or I disagree. Once they get responded this way, you see your avatar moving towards the group of people that share similar views. So, you see, for example, initially, there’s like four different fragmented clusters and these clusters represent the pro-Uber, pro-taxi, pro-rural plays, or pro-transportation law camps. And so, if you click into each one camp, you can see what unites them and then we give what’s called bridging bonus.

So, even within one group, if you can propose some statements that convince that group, then your statement flows to the top. On the other hand, if it bridges longer distance across groups, then on the global leaderboard, your message goes viral. So, basically virality is determined on how much distance do your statement cross to the other side. And so, this became essentially the same algorithm as community notes previously called Birdwatch on Twitter.com but now adopted by YouTube and Facebook and X.com.

But actually it’s a profound algorithmic idea because I think what most people don’t understand is that underneath whatever the system is that makes things go viral is some logic. Does it stir a certain kind of emotion or what is the outcome the AI is pointing at that controls that? And what you’re saying is that if you optimize for bridging, you get totally different outcomes.

But Audrey, can you explain what you mean by bridging?

Certainly. So, bridging happens when two or more groups of people who usually don’t agree on the same set of statements—I think this is great, you think this is very bad—and then along comes one statement that we both think is very good. So, the left wing and the right wing both give thumbs up and something flows to the up wing, that’s bridging.

How did the state implement that? It wasn’t as though you got control of all the private companies and said, “Hey, private companies, do this bridging.” You’re running a state social media system and, normally, when the state runs a product, the product is terrible. Usually, that’s why the private sector wins. How did you square that circle?

We made sure that the infrastructure is maintained by civil society so it’s not exactly the state’s employees coding up the systems. In Taiwan, the Reddit equivalent, the PTT, which is the bulletin board system that most people use is run by the National Taiwan University Student Club for 25 years and it’s all open source. It needs no shareholders or advertisers because it’s funded by academic budget. But because it’s in the campus on academic budget, the state does not have overreach, it’s double arm’s length from the state, and because all the president graduated from that university, it’s also not partisan.

So, the idea here is that the civil society, including academia, already knows how to run such systems so the state provides ample funding but without the top-down micromanagement that would have sunk most of those civic infrastructures. And that is how we do the BBS, how we do the people-led wallet system, contact tracing system, mask rationing and visualization system, all these started their design in the civil society and then amplified by the state.

And is the Taiwanese government still using the system to get feedback and to understand what the people want and how to think through decisions?

Yes. And I described the system as it was 10 years ago before Generative AI. So, it was using AI, the clustering was called k-means clustering, the visualization was called Principal Component Analysis but it’s good old-fashioned AI, it’s not generative. What it means is that it acts on very simple signal, upvote, downvote. But now we have language models that can work on qualitative signals, that is to say it actually understands what you mean. Last year, we used the upgraded version of the system to talk about something very important which is the heart from Generative AI, namely deepfakes.

In March 2024, people in Taiwan were really tired of seeing scam advertisement on social media featuring, say, Jensen Huang, the Taiwanese guy running NVIDIA, trying to give you some crypto or some stock. And if you click, Jensen actually talks to you but, of course, it’s a deepfake voice and people generally felt that they’re crowding out the actual useful advertisements on social media.

Shouldn’t you know that’s fake because Jensen’s already made $100 billion and the guy doesn’t need to sell crypto?

I know, and they always frame it as saying, “I’ve earned this much money overseas, now I want to share it with my country people.” Previously, it’s voice-only because the video generation wasn’t that good but, very quickly, they solved its six finger problem or whatever and it looked very convincing.

Anyways, people also saw those deepfake ads giving Facebook and YouTube a lot of advertisement money, so it’s a perverse incentive because they can afford to pay more and so they dominate the advertisements on those social media networks. So, we are in a bind because Taiwan, for the longest time, was, according to Freedom House, the most free on the internet and so we don’t want the government stepping in and censoring any content. We cannot become authoritarian in fighting authoritarianism or fighting deepfake fraud.

So, what to do? We sent 200,000 text messages to random numbers around Taiwan from 111, the trusted number, and asking, again: How do you feel about information integrity online, what should we do together? So, people share their ideas and thousands of people volunteered to join an online citizen deliberation, what we call alignment assembly, to set a boundary of how AI enters our society.

So, we chose 450 people, stratified random sampling, meaning that these people is a macrocosm of the Taiwanese society, and then these demographically representative people grouped in rooms of 10 in 45 rooms and each room started brainstorming one aspect of this information integrity problem.

For example, one room said let’s make sure that all scam advertisement are defaulted as scam, meaning that, if it featured Jensen, then you need actually to get a digital signature from Jensen saying that, “I’m Jensen Huang, I approve of this message.”

This is insane. So, you basically have a 450-person focus group chosen randomly by the Taiwanese government to advise the Taiwanese government through a bunch of Zoom rooms?

That’s exactly right.

Okay. So, that’s bananas. All right, so these 450 randomly selected Taiwanese people, some of whom have gotten home from work, they’re taking care of their kids but they’ve got ideas on how to stop deepfakes. So, they come up with a bunch of ideas, presumably you have AI synthesize all those ideas and then you bring the ideas to the 450 people, you have them vote on them?

That’s exactly right. In real time, people can see the transcription, the room itself is an AI so, if somebody is too quiet, they nudge this person to speak up. If somebody want to disrupt other people, they can only do so in five seconds.

Wait, wait, wait. Okay, you have these insane Zoom groups of people randomly selected, you have AI that’s monitoring for quiet people so to make sure that all the men aren’t talking over the women or all the women aren’t over talking the men but whatever it is, more likely the former, and you’re able to nudge it so everybody participates. And then, if somebody is too loud, your AI?

Can say that, “Oh, you’re interrupting other people,” but only for five seconds.

Okay. So, you do this, you have this great conversation, everybody talks, they get along, they get a bunch of ideas, everybody votes on them, you then see … The ideas are sent back to who? They’re sent back to the government?

So, in the middle of this almost an entire day of conversation, there’s a plenary session. So, the idea is that people first watch some scientists talking about this problem, policy experts, and then they start deliberating very concrete solutions. One room came up with this idea that we currently, at the time, fine Facebook if they don’t comply and they’re like, “They’ll just pay the fine. Instead you should make them liable.” So, if somebody gets scammed out of $7 million by an unsigned advertisement, Facebook should be liable for the entire $7 million. And another room was like, “Oh, but Bytedance TikTok did not have a office in Taiwan. If we make them liable, they can just ignore us. What to do?” And they said, “Oh, we can slow down connection to their servers gradually until all their business goes to Google because TikTok video will become too slow to load.”

And none of these are censorship, because the censorship idea simply would not be resonant in a room of juries. So, the citizen juries drew what we call the Overton Window, the accepted policy space, and these were synthesized by a language model and read back to those experts in the plenary session who talk about the feasibility, how to implement it, and then people have another round of talk and then they vote.

We can show to all the legislators, regardless of your gender, place you live, your occupation, your age bracket, more than 85% of Taiwanese people in all different groups all agree with this bundle of measures that we determined with multi-stakeholder conversation, checking with the big tech in April, and then we published that in May as a draft legislation; it was passed in record speed in June and July.

And so, this year, starting January, when you start scrolling Facebook or YouTube and you’re in Taiwan, then you don’t see any fake advertisements anymore.

That’s insane. Thomas Jefferson would be very proud. It seems like it might possibly be better than a system where you just have a bunch of highly paid lobbyists pay a bunch of congresspeople and then they decide what they want to do. Maybe?

It sounds like that. And also much more legitimate, wouldn’t you think?

So, you can solve an issue that is fairly easy to grok which is, “Hey, they’re deepfakes of Jensen, he’s trying to give you crypto.” What about a really complicated one? Could you set tariff strategy from scratch using this system?

Yeah, that’s a very good question. For example, during the Uber consultation, we did not say, “Oh, let’s deliberate about the future of sharing economy.” That’s like tariff conversations because it’s so complicated, and it’s also very difficult to pin down to specific feelings because people are experts in their own feelings but only if there’s a very specific scenario. And so, we’re always deliberating just a sliver of the total topic in very concrete scenarios; that’s what’s required for it to work. And we’re very happy, actually, for people to set the agenda also, so people also have this online e-petition system where 5,000 people can tell us: Here is a specific scenario and I think you should deliberate that. For example, the young students, even before they turn 18, are very frequent users of that platform and they say, “We want to go to school one hour later because studies shows that one more hour sleep actually gets you a better grade,” and they eventually got that after this deliberation over Zoom.

We’ve actually had an experiment along these lines, as far as I know, the first one in America—in Bowling Green, Kentucky with your friend Yasmin Green and Google’s Jigsaw project, where they did a version of this and they went out to Bowling Green and they said, “Here are a bunch of issues, everybody talk about them.” AI synthesized the conversations and the ideas and then people voted up or down and now the Bowling Green government will look at them and make decisions on whether they make, I don’t know, the north part of Bowling Green into a historic division.

My favorite suggestion was one person who said Bowling Green should mandate that, in every drive-thru, ketchup is given to every person. Didn’t get a lot of support but it shows just there are pain points that you may not know about, Audrey.

I know.

It sounds like what they did in Bowling Green is very much a stepping stone to what you did in Taiwan.

Yes. It’s exactly the same software, it’s also Polis. Anyone can plainly see that millions and millions of voices nevertheless cohere into very clear, qualitative descriptions, summaries, and each summaries are grounded.

Let me ask you then a bigger question about this. So, my perception is that all of this has made Taiwan’s democracy stronger. Across the world, all the evidence is that technology has made democracy weaker. It has helped authoritarians, it has broken apart democracies. Is the reason it’s worked in Taiwan, maybe I’ll give you three options. Option one, Taiwan is a very small relatively unified country with a gigantic enemy that wants to destroy it so they pay attention to this stuff. That’s option one. Option two, Taiwan had this revolution in 2014 where people came to power and all these saintly nerds took control. Option three, it’s actually just a bunch of software choices and, if the US had chosen other software choices and had set different incentives on different platforms, you could do it too. So, I’m curious whether you think it’s one, two or three or the weights of them?

Yeah, now we have not just Bowling Green, Kentucky but also California. In April 2025, I was in Los Angeles, California with Governor Newsom and his team and we reviewed the first results from the Engaged California platform at engaged.ca.gov. It was about wildfire recovery ideas from Palisade and Eaton areas and in conjunction, again, with civil society leader like the Department of Angels and so on. And again, in record numbers, I think almost 30% of people who saw email saying that, “Oh, let’s give your ideas, your feelings about wildfire recovery,” actually clicked through and gave ideas. So, thousands and thousands of Californians had conversations among themselves setting the agenda about, say, the community spaces for mental health help, so on and so forth and we, again, saw exactly the same dynamic.

People who were initially very polarized came to the middle and agree more than 85% on a set of very concrete measures. And on those measures that still are divided, then they can arrange additional deliberation. So, I don’t think it is a function of the person or the individual or the culture or circumstances or the fact that pressure makes diamonds but I think it’s really just software choices.

Let me ask you about AI. So, AI is clearly helpful in this project. Would you ever trust an AI to actually make decisions? To take all the info from Bowling Green and then just say, “You know what? We’re going to do this.” Will we get there? Do you want to get there?

Yeah. The accurate summarization is really the very first step. Only when people can verify that the summarization works in a way that people can audit repeatedly instead of sending all your data to mainframe and just trusting the mainframe operators vertically in super data centers doing the churning of the mathematics. Nowadays, people can download those models with the same downloadable data. As in Bowling Green, you can go to whatcouldbgbe.com and download all the data and then run exactly the same algorithm and see for yourself whether that summarization is biased or not.

So, I think all this horizontal diffusion of smaller models that are, nevertheless, still trustworthy needs to happen before we can imbue with it more agentic delegation.

So, in order for AI to really be able to make decisions, we need smaller AI that people can run on their own laptops so they can test and verify that everything was done correctly in the big model. So, Bowling Green came out with this report and there are 100 policies that are recommended and you can sort them by the ratio of up votes to down votes and the number of votes which is a proxy for interest. You could imagine a next sorting step where you took a highly trained AI and you said, “Okay, assign a cost to each one of these,” and then you give it a budget and then you say, “Based on these preferences, what are the six to 10 things the Bowling Green government should fund next year.” The step after that would be to just automate it and say the default is that what the AI says we should do is what we’re going to do.

Yeah, I can do you one better. The extreme step would be each of us just train an avatar, a digital twin which knows all our preferences. And so, instead of going into those rooms of 10 to deliberate or to attend a jury system, we just send our avatars and they will deliberate among themselves and make decisions and going back to us. And because they’re very good at persuasion, convincing us whatever they came up with is actually the collective will. That’s the avatar state. It’s like sending our robots to the gym to lift the weights, to run a marathon for us, it’s all very impressive but, of course, there’s no muscle that’s grown this way.

So, the part where I am somehow able to get all of my opinions to weigh in in a complicated conversation without me having to take eight hours in the Zoom room, that’s appealing. Could we get there?

Yes. I think something like a conversation between people of different communities, for example, people sometime already have gatherings where they talk about public policy issues in their civil society. These happen by default, it’s not like the state actually have to pay you to serve your duty to have a conversation about the most interesting thing in technology, on social media, on LinkedIn, we’re doing this nevertheless. So, the point is that we can then turn those conversations into what’s more binding. That is to say, if they satisfy a certain threshold, for example, people do have to sign in with personal credentials, so we don’t know who they are, maybe no need for real names, though we do need to know that they’re real people and living in this policy, maybe that’s the absolute minimum. And then this online conversations can then inform the same set of forums and gets AI summarized and then those AI summarizations then close the feedback loop.

In fact, in Tokyo, there’s a 33 years old, Takahiro Anno-san, who run for governor using exactly this stack. He used Talk to the City, which is a generative AI summarization tool for social media to monitor anyone who want to contribute platform ideas to his campaign. And he decide to run a month before the governor election, he has no party, nobody have heard of him, he just read this book called Plurality, which I co-wrote, and decided to run. And then he made a AI avatar of himself livestreaming 24/7 and so you can literally call into a line and talk to an AI voice clone of Anno and suggest your platform. And at the end of the day before election, independent think tanks ranked his platform, the crowdsource platform, as even better than Koike-san’s platform who, nevertheless, won the re-election of course but now Koike-san taps Anno-san to become the advisor to GovTech Tokyo to apply this what they call broad listening system to keep the feedback loop going between the Tokyo people and the city government. So, a version of this is already happening.

More with Audrey Tang after the break.

[Break]

So, you’ve sometimes talked about the vertical AI race and the horizontal. Explain what those two things are and why this concept matters.

So, the vertical race is the one that we’re all very familiar, people compete to build larger and larger data centers and in those data centers they train larger and larger models. And these models, at some point, figure out how to do programming, how to solve mathematics, and participate in the training of the next even bigger models and it’s called recursive self-improvement, meaning that agent version zero trains agent version one, trains agent version two. And initially, it requires a lot of human researcher and engineers in each step of training but, as the agents become much better at coding and mathematics, they take over more and more of the training of the next generation until such a day it requires very little intervention from human. Maybe they assume robotic body that’s also took care of all the physical parts of this process and then you can get explosive growth, what people refer to as superintelligence.

It’s what we’re seeing right now and that’s Elon Musk versus Sam Altman versus Dario Amodei versus all the folks who run the biggest AI companies. It’s fascinating, it’s crucial, it’s important, and it’s terrifying. So, that’s the vertical race. Now what’s the horizontal race?

The horizontal race is something else. It’s, as I mentioned, the ability for people to inspect smaller models, the diffusion of models that used to only run in supercomputer in data centers but still solve real problems like summarization and translation we just mentioned and allows people to peek into the model itself so that people can interpret what’s actually going on.

So, for example, in summarization, what’s important is for it to not hallucinate. If it needs to generate a headline from a news article, it should not say things that’s actually not in that news article and then, if the model hallucinates, it turns out that its thought pattern is different as when it’s copying one part of the news article. So, if the model runs locally on your laptop, on your desktop, then, unlike a supercomputing data center in which you do not have access to the inner working of the model weights, in your own computer, of course you have access, so you can start instrumenting, you can start looking at its hallucinating patterns. And then a lot of people would be interested in figuring out what does it mean for a model to hallucinate, what does it mean for a model to make things up. And then they can contribute to the horizontal race which is getting more and more people into their hands the tools to inspect models, to see whether they’re hallucinating, whether they develop some harmful intent, whether they’re scheming and so on.

All right. So, the vertical race is these big companies with zillions of dollars building these models going towards superintelligence. Horizontal race, smaller, simpler, open source models and a whole bunch of developers like these communities you have in Taiwan and elsewhere working to build good things into it.

But now, if the big AI companies are able to make recursively self-improving superintelligent models, if big company A creates a model that can then invent the next model and the model after that and then they’re so smart, why is it going to matter if we’ve got six tinkerers in a CS department at Swarthmore trying to figure out how to do identity verification because they’re going to be working on something that’s a billion years behind what the big company has built?

So, one crucial thing that the biggest companies still have not figured out is that, in this race of vertical take-off, currently, nobody has an idea. A version of this agent, when it’s training the next version of the agent, an even bigger model, whether it’s training the successor to be loyal to humanity, to the model specification, or to the constitution or whether it’s actually scheming and training a successor to be loyal to itself.

Because, at some point, these very large AI models may evolve a very different motivation than the one that we trained into it and, currently, the science isn’t good at all in removing all these incentives. The danger then is that, at some point, humanity still thinks we have control but these superintelligent agent actually already assumed control. For a very detailed exposition and a scenario, just consult ai-2027.com.

Yeah, that’s a terrifying document. But isn’t that why the horizontal race might not matter? If that happens, if the scenario in the AI-2027 doc where you have these all-powerful self-recursion agents that are not aligned with human interests, then who cares what’s going on in the horizontal race? It’s like fixing bicycles to go on a road where there are Lamborghinis.

In 2027, they answered this question because, at a very critical juncture in training the next probably misaligned agent, they stop, they paused and then they look around to the community to ask all those bicycle tinkerers is anyone working on the same architecture that has what’s called mechanistic interpretation tools that can help us to look at this half formed agent and figure out whether it is misaligned.

So, this is what you’ve called summoning the Buddha. So, your expectation is that the folks building this massively powerful super intelligence are going to say, “Oh, boy, we’ve built a dragon and the garage door is still down, maybe we shouldn’t open it, let’s go ask some people if they have good ideas for what we can do with this dragon?”

That is exactly right. And it turns out, if this dragon shares the same neural architecture as the smaller one just as Gemini shares the same architecture as Gemma, then the people who work on Gemma to figure out how to remove its self-preservation, their ego drive and so on, the same technique can apply to the larger, much larger Gemini model as well.

Okay. So, this metaphor is about to go insane, Audrey, but I hope you’ll hang with me. So, big companies, they’ve built this giant dragon. Small tinkerers, they’ve been working on these small dragons, they figured out interoperability, they figured out human verification, they figured out how to stop CSAM, they figured out alignment, they figured out all these things that are really important. Owners of big dragon come to folks in the horizontal race of the little dragons, tools are then inserted and then dragon turns into Buddha?

Yes, into a being with Buddha nature.

So, now, this is dependent on two things. One it’s dependent on the horizontal race being able to keep up and then, two, it’s dependent on whoever controls the garage door behind which the dragon sits deciding that it’s time to call in the tinkerers as opposed to letting the dragon out.

Exactly. Dario Amodei of Anthropic recently posted an essay called “The Urgency of Interpretability.”

That was a wonderful essay and he said that he wants more tinkerers at the big companies but he also wants the government to fund more tinkerers. He also said we need to make sure that we slow down China’s AI industry so we have time for the tinkerers to do all of this.

Well, if one side is summoning an extremely different configuration, different architecture that is actually not compatible with whatever the horizontal diffusion the personal computing geeks and tinkerers are working with, then it’s hard to say that it can actually apply so cleanly to that summoning.

Do you think it would be better if the US AI industry and the Chinese AI industry were more integrated or do you think it’s better that they’re going in separate paths?

If people see that, in order to deploy AI in a believable, trustworthy fashion, that inspectability really is very important, then I do feel that it is possible instead of just running all the leading AI on supercomputing data centers. What Jensen Huang said is a personal supercomputer becoming a viable alternative so that people who run those GPUs in smaller Mac mini-like configurations can then run those smaller models, and even if there are different architectural choices, even if there are trap doors in models such as DeepSeek, then people, the tinkerers, as people in perplexity have done, they can analyze how the model works. So, yes, if the model weights are released in the open, this kind of collaboration becomes possible.

When DeepSeek came out, how did you react emotionally? And by that I mean it was both a wonderful fulfillment of much of what you’ve been talking about, which is an open source model made widely available. But on the other hand, it was a huge technological achievement from a company in a country that wants to destroy yours.

For me, because I’ve been looking at Hugging Face conversations around V3 that was even before R1 and the preview edition, I tracked the contributions they made on top of existing architectures like Mixture of Expert which was pioneered by Mistral and, among others, Llama and so on, which are French and US respectively. So, it’s okay to figure out how to overcome the early chip exports and also figure out how to put another Lego block on the top of Lego Tower but each Lego block in that tower was from France, from the US, some from Taiwan. And so, in a sense, yes, it’s validating, it shows that open source can actually make reasoning work in even an edge configuration.

Very quickly after DeepSeek-R1 gets released, as I mentioned, not just the perplexity version but also other open source versions very quickly overtook it on the leaderboards. So, it is technically quite interesting that they figure out how to circumvent the export control to that degree but, in absolute quality-ish comparisons, there’s better choices both open source and non-open source compared to DeepSeek-R1.

So, you actually weren’t that impressed with DeepSeek?

I’m impressed that they’re able to overcome the handicap to figure out how to connect those chips because those chips were restricted and they wrote very low-level code to overcome that restriction. But then it’s not exactly the state of the art for very long, it’s state of the art for maybe one week.

How do you think about the risk to Taiwan right now? You help make a country stronger, that is at great risk with China, and you work in an industry that is the linchpin of this conflict because so much of the decision-making around whether China will attack Taiwan is around tech and it’s around the chip manufacturing capabilities of TSMC and the brilliance of the Taiwanese chip manufacturing. What do you think is going to happen between China and Taiwan?

We’ve been dancing this tango for a very long time. In 2014, the Sunflower Movement that I participated in and helped livestream and facilitate saw half a million people on the street and many more online deliberating should we allow Huawei and ZTE into our then new 4G infrastructure, should we allow the investors from Beijing and Shanghai to run communication platforms, journalism newspapers in our publishing industry.

So, the service trade agreement would have opened the Taiwanese communication information infrastructure and ecosystem to the investors from Beijing and Shanghai, and people summarily said no. I don’t think there’s any serious party in Taiwan that still harbored the same hope because we have seen what happened to Hong Kong after the national security law.

Yeah. I think the most cherished change of address form that we have at The Atlantic came from Madame Chiang Kai-Shek.

Oh, wow.

And she sent a letter and she said, “Just so you know, I have a new address. Please mail The Atlantic to my home in Taiwan.”

Wow. Yes, exactly. And so, in having solidarity among our political parties in a common rejection against the Hong Kong model, nowadays, you’ll see rallies in Taipei where the ROC flag that Madame Chiang Kai-Shek very much liked and the Taiwan island flag used to never fly in the same rally, nowadays flying in the same rally. Because no matter whether you think Taiwan is ROC Taiwan or Taiwan ROC or was the other way around, nowadays we’re in solidarity when it comes to counter authoritarianism without getting more authoritarian ourselves.

I want to ask one last question that I ask all of the guests on this show. What would it take for you to be convinced that AI is conscious or self-aware?

An AI agent earns the badge of consciousness only when a science-briefed, randomly selected citizen’s jury deliberates on transparent evidence and judges it worthy of the moral standing that we give sentient beings.

That is an amazing answer. So, instead of defaulting to philosophy, you are saying that democracy should choose when something has reached a level of consciousness.

That’s how democracies decide to give more people the vote.

We have to have a discussion beforehand on what consciousness means.

Exactly, because the definition of democracy changes over time. When democracy was first invented, only very few percent of the population actually enjoy the voting rights back in Greece. And then now we see also people gradually expanding the right of other people of different gender, different ethnicities and so on to vote across the centuries. And so, it’s always the same process: a science-briefed, randomly selected jury determines. And sometimes, of course, it’s not so randomly selected but I happen to like random selection.

Thank you so much, Audrey. It has been fabulous to talk with you.

Thank you. Live long and prosper.