Thanks for the thoughtful conversation. Live long and prosper!

Wonderful — please send it along.

<strong> OpenAI </strong> originally adopted a <strong> capped‑profit </strong> model under a <strong> nonprofit </strong> — a Mozilla‑like structure — though governance evolved with investment. The broader point stands: if <strong> defense‑dominant openness </strong> does not raise risk, policymakers can require <strong> interoperability </strong> and <strong> shared safety baselines </strong> ...

There is a <strong> precedent </strong> . <strong> Netscape </strong> seeded the <strong> Mozilla </strong> public browser effort; the <strong> Mozilla Foundation </strong> stewarded open web standards that enabled Firefox, and the ecosystem prospered (Chrome, Safari, etc.).

To remedy this, I support <strong> Public AI </strong> : a <strong> ICANN‑like </strong> network producing <strong> permanent public safety goods </strong> , <strong> publicly accessible and accountable </strong> (not necessarily state‑owned). Regulators can require frontier labs to <strong> co‑invest </strong> and <strong> interoperate </strong> — similar to how telecoms ...

Even the strongest critics — e.g., <strong> Eliezer Yudkowsky </strong> and colleagues at <strong> MIRI </strong> — call for <strong> more </strong> work in <strong> mechanistic interpretability </strong> , <strong> corrigibility </strong> and <strong> explainability </strong> . The issue is the <strong> gap </strong> : <strong> an order of magnitude ...

Gradually, yes. At <strong> Bletchley Park </strong> the debate was intense, so we co‑drafted an <strong> Openness and Safety </strong> statement with <strong> Mozilla </strong> and others. Already at the <strong> Paris Action Summit </strong> we saw a <strong> narrow corridor </strong> open between “brake” and “accelerate”; <strong> ROOST </strong> ...

In practice, leaders in both camps prioritize <strong> epistemic security </strong> and worry about <strong> malicious AI swarms </strong> , <strong> coordinated information harms </strong> and <strong> synthetic harassment </strong> that chills speech. The overlap is the <strong> steering wheel </strong> : <strong> peaceful conflict resolution </strong> , <strong> robust ...

Yes: beware of <strong> hallucinated polarization </strong> within the AI community — “doomers vs. accelerationists.”

I am <strong> optimistic </strong> . Many democracies are at <strong> peak polarization </strong> ; people in multiple parties are tired of <strong> extremes </strong> dominating the megaphone. AI can <strong> scale broad listening </strong> . Traditional polling collapses nuance into a Likert tick; <strong> deliberative, generative polling </strong> — ...

As routine tasks automate, workweeks likely <strong> compress </strong> (five to four to three to two days), freeing time for <strong> civic engagement </strong> , <strong> community </strong> and <strong> spiritual </strong> life. People will seek meaning in <strong> co‑creation </strong> rather than zero‑sum competition. Taiwan’s <strong> 2019 curriculum reform ...

Less routine work; more civic meaning and participation.

I have said publicly (e.g., in an interview with <strong> Nick Thompson </strong> at <em> The Atlantic </em> ) that I would accept claims about AI <strong> consciousness </strong> if a <strong> randomly selected, well‑informed citizens’ jury </strong> , using <strong> transparent evidence </strong> , deliberated and chose to <strong> ...

People will increasingly attribute <strong> experience </strong> — even <strong> qualia </strong> — to AI agents; some will argue for their <strong> consciousness </strong> and <strong> moral standing </strong> . As <strong> vertical authorities </strong> (ministers, scholars, journalists) lose automatic deference, because anyone can summon agents that <strong> speak with ...

<strong> Trust re‑roots horizontally </strong> as institutional authority weakens.

In the near future, we move from <strong> input‑output </strong> to <strong> experiential </strong> AI agents. Systems will not wait for prompts; they will <strong> explore </strong> , <strong> collaborate </strong> and <strong> accumulate experience </strong> . Much current progress — e.g., <strong> coordination for autonomous robots </strong> (avoiding collisions) ...

Today, AI systems <strong> infer from data </strong> to generate <strong> predictions, recommendations, content or decisions </strong> that <strong> affect an environment </strong> — a broad definition consistent with the <strong> OECD </strong> , the <strong> EU AI Act </strong> and recent U.S. policy language.

<strong> Care ethics </strong> addresses <strong> radical asymmetry </strong> between caregiver and cared‑for. As <strong> Geoffrey Hinton </strong> suggested with the “maternal instinct” metaphor: a <strong> gardener </strong> moves far faster than the <strong> plants </strong> , but chooses to act at <strong> plant speed </strong> . Imbuing AI with ...

Humanity already forms <strong> super‑intelligences </strong> through institutions and corporations. AI agents add <strong> bandwidth and speed </strong> — e.g., one “mind” with <strong> many bodies </strong> . This breaks simple <strong> utilitarian </strong> calculus (machines can optimize at scales and speeds that dominate the utility ledger), and <strong> rights‑based ...

I advocate for an <strong> ethics of civic care </strong> multi‑agent futures with radical asymmetry.

The PRC will likely pursue standards leadership — its <strong> 5G </strong> strategy is the playbook. The U.S. <strong> Clean Network </strong> initiative previously checked <strong> Huawei/ZTE </strong> in core infrastructure across allies. In AI, however, the gap is <strong> smaller </strong> : the U.S. has <strong> more compute </strong> ...

Expect a <strong> standards </strong> contest, but less asymmetry than <strong> 5G </strong> .

Some labs in the Bay Area still frame a <strong> vertical </strong> race to <strong> superintelligence </strong> , but policy momentum — and needs outside a few hubs — point to <strong> diffusion, interoperability and local steerability </strong> .

Most countries want <strong> steerable, “no‑strings‑attached” </strong> technology they can <strong> govern locally </strong> . <strong> Open, interoperable models </strong> and tooling set norms — much as the world standardized on <strong> TCP/IP </strong> . That is why I emphasize <strong> defense‑dominant open safety </strong> : shared safety tech makes ...

The main race is <strong> horizontal diffusion and standards </strong> , not a vertical dash to a singleton.

On OpenAI specifically: my understanding is they <strong> delayed </strong> releasing certain <strong> GPT‑OSS </strong> models until they could reduce <strong> offense‑dominant </strong> misuse in <strong> bio </strong> and <strong> cyber </strong> . When open releases <strong> do not </strong> materially raise those risks, commercial and open incentives can align.

Open releases of <strong> previous‑generation models </strong> enable research on <strong> steerability </strong> , <strong> mechanistic interpretability </strong> and <strong> scalable oversight </strong> . Frontier labs can then adopt those improvements. <strong> John Carmack </strong> at <strong> id Software </strong> open‑sourced prior engines not out of pure altruism, but to ...

The tension is real but <strong> manageable </strong> — <strong> and often productive </strong> .

That logic informed <strong> ROOST </strong> — <strong> Robust Open Online Safety Tools, </strong> which I launched with <strong> Yann LeCun </strong> and <strong> Eric Schmidt </strong> at the <strong> Paris AI Action Summit </strong> . Take <strong> synthetic CSAM </strong> : a <strong> federation </strong> of mid‑size platforms (e.g., ...

The <strong> 1990s cryptography wars </strong> taught us that: strong crypto (e.g., <strong> PGP </strong> ) was once treated as a munition under export controls, yet secure practice ultimately came from <strong> open algorithms, open reference implementations, responsible disclosure and broad red teaming </strong> . Today even <strong> post‑quantum cryptography ...

In <strong> defense‑dominant </strong> domains — when it is easier to defend than to attack — <strong> openness increases security </strong> .

For reference, I am developing a microsite called the <strong> “Six Pack of Care” </strong> — practical components for deploying civic‑care systems at scale.

Polarization, manipulation and degraded shared reality are already her — often amplified by <strong> misaligned recommender systems </strong> , <strong> synthetic media </strong> and tailored persuasion. My reframing is: do not stop at <strong> harm mitigation </strong> (where “success” looks like nothing happening). Build <strong> pro‑social sense‑making </strong> that measurably ...

The utilitarian lens has limits, but it is right about <strong> immediate </strong> catastrophic risks to <strong> epistemic security </strong> .

I am presently working in Oxford’s <strong> Ethics in AI Institute </strong> (distinct from the <strong> Future of Humanity Institute </strong> ) as an AFP fellow and continue conversations across these communities.

1. <strong> Utilitarian / existential risk and benefit. </strong> Strong in Silicon Valley and Oxford‑adjacent work — <strong> Nick Bostrom </strong> , <strong> Toby Ord </strong> , <strong> Anders Sandberg </strong> , and others — framing <strong> x‑risk </strong> and <strong> long‑term benefits </strong> (e.g., <em> Superintelligence </em> , <em> The ...

I see <strong> three live traditions </strong> :

It is not only human–human; we increasingly see <strong> human–agent </strong> , <strong> agent–agent </strong> , and even <strong> human–animal </strong> mediation via AI. We explicitly avoid framing AI as an end state like a “technological singularity.” The very name <strong><em> Plurality </em></strong> was chosen to contrast with <strong> “Singularity” ...

We emphasize <strong> attentiveness </strong> , <strong> responsibility </strong> , <strong> competence </strong> , <strong> responsiveness </strong> and <strong> plurality </strong> connecting them so that conflict becomes <strong> fuel for co‑creation </strong> . AI amplifies both the <strong> breadth </strong> of conversation (translation, summarization, bridging across communities) and its <strong> ...

The book centers on <strong> civic care </strong> ; AI is instrumental to achieving it.

I actually entered the field early. I started programming in <strong> 1989 </strong> . The <strong> Tiananmen </strong> crackdown shaped my family’s attention to democracy and autonomous communication systems — my father’s thesis explored those dynamics. After the <strong> Berlin Wall </strong> fell, he went to Germany for PhD studies; ...

We pioneered <strong> sense‑making and bridging systems </strong> : qualitative aggregation of public preferences into <strong> “good‑enough group selfies” </strong> that allow the public to steer policy. We used those methods on issues like <strong> deepfake regulation </strong> and designed <strong> codes of conduct for AI agents </strong> entering society. ...

I served as Taiwan’s first <strong> digital minister </strong> — sometimes described as a “cyber ambassador,” where <em> cyber </em> comes from <strong> kybernētēs </strong> (Greek for “sea captain”) — and my focus was using AI‑augmented tools to help society coordinate, with over <strong> 7 years </strong> in the Cabinet, ...

My work is about societal <em> steering </em> of AI to foster cooperation.

And thank you for agreeing to donate this transcript to the public domain. For context, I’ve published <strong> 2,000+ public meetings </strong> over the past decade, so I am very comfortable with open publication.

Yes, literally in the future. Doing well, thank you — and yes, please go ahead and record.

Of course. Thank you for your time.

We can negotiate, but usually I make a textual transcript, lightly paraphrasing, send it to you to co‑edit, and once we are both happy, we publish it to the public domain.

Sure; I am in the U.S. every other month. I am also an AFP Fellow in Philosophy at Oxford along with Alondra Nelson, working on machine ethics — training multi‑agent systems to practice civic care. That sits at the intersection of CS, politics/mobilization, and philosophy — natural allies for CITP.

As for Yarvin’s CEO‑king idea: the DOGE experiment shows that unchecked CEO power yields mixed results. Public experiments are valuable; we learn by trying.