Yeah, I would use the EU term, information manipulation.

And so with open weights models, and local fine tuning, watermarking only works when it’s like a book length, but the information manipulators don’t write books to do their manipulation. They rely on interactive, deep faking. The trust is gained by, in real time, they can synthesize a response that ...

With Llama and local fine tuning, one can easily rephrase whatever gpt four says in the style of communication that I use on my emails; there really is no way to discover the watermark anymore. I think up to, I don’t know, a few hundred tokens or whatever. It’s been ...

Yes. So watermarking relies on language model talking in a specific way with an accent that is not quite human in terms.

Yes. The day I saw the Llama weights leak on torrent sites, I knew watermarking was doomed. There’s no hope for any watermarking in LLMs.

The metadata.

And so, by reducing this problem to an authenticated actor problem, we first say, okay, from this point onward, the government will only use three or four digits code as a sms number to communicate with you, and we will never use ten-digit code. This is basically saying only the ...

And we’re already at a point where the local tax agency earlier last year sent a sms encouraging people to file tax online to reduce carbon footprint and whatever, and just give like a scam so the trust on the provenance of those messages are no longer there, when everybody ...

Our main defense is on the actor’s side. So starting this week, actually, the governmental agencies will share this 111 number in the sms messages that we send out. Previously, many agencies, especially local municipalities, used their own numbers, but as they proliferate, it’s impossible to know if this comes ...

It means that anyone can call me very easily.

Right. And if that someone has published into public domain [laughter]

Previously, you have to speak pretty good mandarin to con someone speaking mandarin and understand a local Taiwanese context. But now you can mount it from anywhere in the world, speaking no mandarin at all, just simply by copying a voice and communication style of someone you trust.

And the reason why is LLMs, or generative AI in general, do the most damage when they destroy the mutual trust that is existing in the society, on the actor level, when you pick up the phone and you hear your friends and family’s voice, but it’s actually a voice ...

So for communication, I usually use the abcd actor behavior content distribution strata. It’s part of the EU disarm framework on fimi, foreign information manipulation and interference. And so just the word fimi, as you can see, no longer talks about the content layer anymore. Foreign is on the actor ...

This is why I find a problem with the disinformation misinformation narrative, because all llm-wielding people don’t. It’s no longer their weapon of choice. They don’t rely on dis- and misinformation anymore.

Yes.

So it’s crowded out. It’s a DDoS. It’s a denial of service in the democratic language, which is always so limited before the election anyway. So then it has the effect of reducing the contextual integrity of all other issues.

Like whatever referendum topic you talk about, whatever you want to have a conversation about marriage equality, and suddenly everybody’s talking about air pollution. And so this kind of persuasion is neither misinformation or disinformation. It’s not even malinformation. It’s just saying that air quality really matters. But the upshot ...

Yeah, with the airbox. It’s indisputably true. If you think it’s wrong, you can buy an airbox for eighty bucks and measure it yourself. Although what’s actually happening is that for many conversations online, and because facebook civic integrity team wasn’t there in 2018, you would just see air pollution ...

On the other hand, you can also start a referendum that bans coal burning plants or whatever, and be on both the referendum ticket, which can be promoted close to the election, while being also on the mayor of the ticket, which cannot be promoted close to the election. And ...

Right. So we’ve learned better. We’ve switched them to alternative years. But back then, it was quite perfect because you can campaign as a mayor or candidate on a platform of, for example, reducing pollution, air pollution to restore the sky of 台中 (taichung), and clean air.

That is right. And also during 2018, because we were at peak infodemic back then. I think that’s because the referendum and the mayor election were on the same day.

So socially, as long as there’s general societal trust, fact checking does work against information that is blatantly false or can be shown to miss important context, but to meddle in the election or to get people to buy something or whatever, to do persuasion. One does not need this ...

Right, exactly. So people do not intentionally transmit false narratives to their friends and families. And fact checking, if it’s done at scale and participatory, does work because people would go out of their way to say to their friends and family that actually I was wrong, that that was ...

But I mean, like earlier on during SARS, even though it’s very toxic, very lethal, it is also self-limiting in a way, because once you get sars, you only get to infect a few people before you become immobile. And covid was so successful because especially omicron in later iterations ...

On the other hand, which may be true if these are artisan like written by people, but the LLMs, because they know exactly how to persuade the effect. And valence. Affect includes valence, the effect of people they do not actually need to fabricate anything that is not already true. ...

Well, just a word. Disinformation itself is a problem. I mean, it’s better than fake news. Of course, nobody uses that word anymore. Yeah, except some. But anyway. But even the term disinformation suggests that the content is wrong or untrue, or at least missing in context.

Right? Exactly. So by making oneself vulnerable, if one would act in a bad faith position in the next iteration, one would strengthen the ability for both sides to find some common ground, even a glimmer of common ground because it’s in both parties best interest. And the good thing ...

Right, exactly. So basically, one would not easily misinterpret the intention of an action because one would then say if it’s trying to do something that is untoward, they have already pre committed a lot of energy that they would lose if they actually mean it the wrong way. So ...

So a confidence building measure I think starts from the cold war is that two parties that have no reason to trust another can nevertheless commit some energy so that the people on the other side would incrementally trust that the actions and so on are interpreted in good faith.

So just like those clusters that I showed, you can see that these are the main tension points and the confidence measure has to appear here. Of course, we didn’t use LLMs to synthesize that line, but something analogous is happening. But to do this systematically and say that anybody ...

Yeah. So by saying that we’re definitely not doing that. By saying that this is not a kinship affecting move, this is what’s called a confidence building measure, cbn. And the way that we kind of produce, synthesize those. It is of course partially assisted by digital participation platforms because ...

Right. So a lot of the cultural debate back then was that do we have to invent like new 「女媳」or whatever? 媳 with a 男 radical. How does that work? Right. And so it is…

Yeah. And some words itself carries a patriarchal assumption like 兒媳 which is the son of wife uses a different word 媳 that is only used for son’s wife.

And it extends like three degrees. So it literally has 16 different words, for example. And also the ordering of the ordering page is.

Like, they both care about marriage, otherwise they would not spend calories debating it. Right. The main thing in Taiwan was just about kinship. One of the most slippery slope arguments we heard around that time was that we would need new words for aunts and uncles.

Yeah. So part of the work that I alluded to is to establish a shared context of what it is not. I know this first hand because my grandparents on my father’s side are catholics. During the 2018 referendum, we had a lot of those conversations and the importance of ...

Slippery slope arguments.

We have existing training materials, classes for bridgemakers of a certain width of director, like, we were taught about confidence building measures, about commercial facilitation and so on, but always with a very kind of fixed conflict still in mind. But this kind of scale-free conflict co-creation, I think that’s ...

And within it, the scope is narrowed. So that it only talks about the things that even, like a catholic taiwanese, would actually not push back on because it’s not challenging any of their existing notions. And so a way to first systematically find it, and a way to foster ...

This is the idea of rough consensus. We produce some running code in this particular instance, the code for the constitutional interpretation. Right. So the act itself doesn’t have a name. The marriage equality act in Taiwan is just the enforcement act of a certain constitutional initiative. So this is ...

Exactly.

Exactly.

And so earlier in the referendum movement, we saw a very conscious, blended movie that says marriage equality in this stage only pertains to the rising duties of two individuals, but we are okay for them not to form kinship with the patriarchal family system, like slicing and dicing a ...

And this is something that you see in a lot of those conversations. So I think what’s missing and what Taiwan has been doing quite well, I would say, is this idea of rough consensus, of this continuous alignment, not related to ai at this time, that we settle on ...

But then it is sometimes criticized as too in group facing and too challenging, like too challenging to explain to an existing patriarchal family hierarchy culture, and so kind of not easy to explain and even harder to culturally transmit. But then it is important to the queer community and ...

Reinforcing a simple narrative that is actually overlooking the deep societal connections that is formed by this kind of family, extended bonds. Whereas people who work on, like, lgbtiqa+ community solidarity, they originally proposed a very complex law that recognizes all sorts of forms of bonding and so on. The ...

Or whatever. Right.

So one would say, for example, that, for example, during the marriage equality movements here in taiwan. If you take a non-taiwanese really, but kind of what happens anywhere in the world, you would say that people who fight for the equal rights and duties, recognitions and so on, as ...

It’s not always branded as that.