Exactly, yes.

This is the main threat model that I’m worried about. So, in cybersecurity, I mean, DDoS is not the most impactful threat. And yet during the Pelosi visit, the thing that harmed us the most was DDoS.

And there are no existing institutions that defend against it, because this is more exponential, right? When criminal organization act in some way, there is a more smooth buildup and you can detect the activity. But if it is just a large number of individuals deploying these agents, which is ...

It doesn’t change the ceiling, but it does change the mass. And this is my second point, which is when there is a mass of sheer number argument, that a mass of people who are dedicated to do harm and that their capability can be amplified through partial automation, then ...

So, people who are dedicated to do harm were previously limited by their institutional access, to such top-level tools. But now with generative AI, they can get almost halfway or 80% there just by interacting with generative AI. So, people who are dedicated to do harm are also democratizing their ...

The effect that generative AI generalized model has is that it lowered the accessibility. Previously, you will have to be a top expert in order to synthesize your deepfake clone. But generative AI makes the ladder of expertise very smooth in that everybody with just a little bit of script-kiddie ...

So, I think it’s the new surface, new attack, the unknown unknowns that we should be worried about, not the proliferation of capability on existing domains. Not because that these are not important, but rather because of the principle of subsidiarity, anything that we can name already probably has institutions ...

Yeah, so the threat actors currently are not relying on GPT-4 because for each narrow threat domain, there already exist specialized models that does it better. And it’s the same for synthesizing fraud or scamming or political manipulation and things like that, simply because it takes time to learn these ...

Exactly.

And so, if it causes harm, first there’s diversity in the narrowing, in the fine-tuning, so unlikely to be subject to the same harm at the same time. But even if they do, it’s in their best interest to switch to an updated model, exactly like you would upgrade your ...

So, we understand that it is impossible to completely ban what’s called shadow IT, right? People just bring in their own devices to work. On the other hand, the kind of Facebook page harm you mentioned only happens when there’s dependence on choke points. However, in an offline edge-deployed model ...

Yes, so for that particular, like over-dependence threat, the generative AI guideline that I shared with you the very first link, the section four, is what we’re looking at right now. So, the section four basically says government agencies and the people they pay may not use gen AI to ...

Oh, I mean, I’m happy to talk about it too, but if you have another meeting to run, too, that’s good.

Right, exactly. So, it’s self-healing and also designed to basically expect that at any given time it will probably fail, and that’s the chance of updating our threat model, basically.

Right, exactly, because they cannot be breached at the same time, right? So, an anti-fragile system sees a vendor, a single layer being breached, automatically updates and so on, and in 24 hours reinforces itself. But it can do so only because the other two layers of different vendors are ...

Yeah, to validate that your defense-in-depth actually works, so that your system stays anti-fragile even when each and every vendor is assumed to be breached at some point.

Yes. So, I think that the idea of “automated continuous validation” really is the kind of unifying meme here. So, like such models exist instead of giving them autonomy, we narrow it, so for each possible domain, they become continuous validation tools. And then we redesigned the infrastructure because we ...

Exactly.

I’m willing to trust them in a staging environment.

So, this capability already exists and is used already in this way, which is why defense in depth is so important, because we have to always assume any single door or any single vendor is already breached by this sort of technology.

And so, their recommended use of that is just to put it in a carbon copy of your actual production environments, into your staging area, and just let it do its thing. And so that you can upgrade to defense in depth, so you can upgrade to better defenses and ...

There are specialized companies, like Pentera, already has this capability. I was in Israel; they did a demo to me. They’re basically the top red teamers from the Israeli Defense Force, battle-tested. And they just made an automated system that is capable of discovering and synthesizing zero days in the ...

Yeah, it’s my job.

So, in a sense, that threat is very much like what we envision those more highly AGI-level threat actors to be. And unlike pretty much any other domain, this domain already understands automated threats.

Right, exactly. And yes, that’s the metric, and this is the structure of threat information expression. And the good thing about these frameworks is that they’re designed to counter the threat actors that are already assumed to be AI in some way, right? Because as a black hat, you would ...

What I’m trying to say is just that when the harms are gradually being assembled and aggregated, as long as it conforms to the format of the MITRE ATT&CK, the cybersecurity institutions in private and public sectors know how to address and coordinate responses if it’s of that format.

So, even though that is not a computer virus, it is a coordinated inauthentic behavior on social networks and such that aims to achieve effects similar to cybersecurity attacks: stealth, lateral movement, and so on. There’s a kill chain, right, in terms of trying to sway people this way or ...

Yeah. I mean, there are a lot of laws in the US. The cybersecurity information sharing… it is not a new thing, right? It’s already existing. The thing was just that they address vulnerabilities of computer viruses in computer systems. And just recently, I think the CISA, the US equivalent ...

So long story short, I believe there are existing cybersecurity multi-stakeholder conversations that are capable of surfacing this kind of unseen harms that lurks for a very long time, like those advanced persistent threats, and that if governments start issuing guidelines on how to use things responsibly with the accredited ...

On the other hand, the great thing about making or tuning our own model is that we can include by default ways of good practices, especially around governmental agencies that do follow the cybersecurity rules. So then exactly the same threat indicator and reporting and red teaming and white hat, ...

And this is natural because there’s currently no widely adopted privacy-preserving way of sharing such harms. So even if all the GPT-4 users are suffering some epistemic harm, or phenomenally, logically speaking, the harm on their everyday lives, there’s no easy way to surface that at the moment on the ...

And these two pillars are important because if a government, sovereign government, do not offer their own foundation models in a democratically accessible way, they’re less informed in a position vis-a-vis the largest AI labs when it comes to industrial use because there is currently no easy way for the ...

So, in the document that I pasted to you, which is a draft of the Generative AI guidelines for our public sector. So, there’s two pillars in the Taiwan strategy, in the Taiwan model. One is that the science ministry, actually makes its own foundation model, named TAIDE. And the ...

Of course.

I know of Daniel, but not personally.

No, I don’t think so.

Exactly, yes.

But the more insidious unknown unknowns is the kind that you mentioned, which is the subjective, phenomenological, plural subjects, the harms that changes the fabric of trust, changes the fabric of society, that don’t currently have an institution in charge of that. And that’s, I think, where we should focus ...

Right. So, my point is that, no, we’re not saying that we don’t forecast those harms. We’re saying that we forecast it with the principle of subsidiarity. The idea is that these institutions are closer to the harms, so they should be part of this subsidiarity scheme where they’re empowered ...

Yes, that’s it.

So, let me simplify the argument. There are financial institutions, there are election institutions, there are nuclear nonproliferation institutions, and so on. These institutions are all very interested in protecting their existing processes. And so, the national security people already have ways to track generative AI harms within their purview, ...

But if we focus on the first, then it becomes just protecting the bureaucratic processes, the existing institutions, and so on. And there’s a lot of ways to do that, but I don’t think that is where we should focus most of our energy on.

So, I think if we focus on the later, it is possible to surface that sort of harm, usually through ethnography, interactive ones like that, and so on, and the question then become how do we scale that sort of ethnography so that everybody can do it, and also the ...

And it looks like that is threatening the status quo when it comes to campaigning, but what people are actually saying is that they previously enjoyed this capability of building personal connections around a social object, that is political speech around political figures, and now that feeling is being decimated, ...

Like, people say that synthetic media will threaten people’s trust in political expressions, and so nobody will be able to tell a campaign speech from a synthetic campaign speech, and things like that.

Right. I think here, we probably need to make the distinction between harming like the status quo, the business as usual, the institutions, and so on. That is to say harm as in disruptive technologies. Vis-a-vis harm to some cherished, phenomenologically speaking, subjectively desirable experiences, immersive experience that people care ...

And then in exchange or reciprocally, the Meta team could say that this is part of “race into safety”, instead of “risks are overblown” or things like that.

Yeah, exactly, exactly. The bridge builds both ways, right? So, LLaMa 2 did a lot of innovation, especially for enterprise environments, on low-cost alignment and really fine tuning for alignment. And so, on this side of the bridge, people who signed that extinction risk statement, including me, would go publicly ...

Right, exactly, exactly. And of Meta not signing these statements, actually.

If Meta, Hugging Face and so on don’t see a way to contribute to race to safety, then they might keep making public statements that decimates the legitimacy of racing to safety.