So, this is already mature, off-the-shelf technology?

The epsilon is always viewed in the inference.

So, just so that I understand the context. When it comes to bootstrapping, this part, is it a necessary enabler? Like, that you have to have this certified LoRA or uncertified process?

Or interactive differential privacy settings…

So, I think the point we’re making in our data altruism guidelines is we stop analyzing privacy solely on the individual level. Because exactly as you observe, it’s almost meaningless for an individual level how much harm, right? Because if I donate or I give up my genomic data, so ...

There are a collective negotiation of a people, specifically for medical research purposes. Tthe idea is that if somebody is willing to buy specific outlying properties from a people, there were some precedence on that, how much price they’re willing to pay in an open, willing foreign market. It’s a ...

So, I think for specific risks and harms, once there is a societal evidence to it, and if the policy makers can very quickly then say, oh, the liability now rests on the intermediary, the ones that are profiting from this AI service, then the platforms actually can invest a ...

But that’s not just an anecdote. We actually changed three laws. Non-consensual intimate images and video, and financial fraud, and also, election meddling, deepfaking candidates. And it’s all about re-internalizing the externalities. So, if Facebook knows that there is a sponsored advertisement, a financial scam, defake or shallow fake, that ...

For the risk side, harm side, one thing is easy to quantify, which is the money lost to financial scams. Because previously it takes a Taiwanese to scam a Taiwanese over the phone. But now everybody can scam a Taiwanese over the phone.

They’re all outliers.

And just like the EU, we have another guideline that says, if a co-op or a non-profit’s purpose to be a data altruism organization, or in our guidance, called a data altruism operator, or something like that, then first it must have no profit motive. This organization must not…, actually, ...

Which is less than true, just like Universal Watson, like in a theater. But into something that’s actually respectable and zero knowledge. Like the zero knowledge proofs we talked about, homomorphic encryption, encrypting the query for the inference to operate in a homomorphic environment. Then responding the results that the ...

We’re issuing two guidelines by the end of the year. One for privacy enhancing technologies. We make a clean delineation between things that are just somewhat privacy preserving, like K-anonymity.

We simply disallow for-profit companies from participating as data altruism organizations.

And coupled with what you said, which is LoRA or fine-tuning processes are far easier to verify and certify, even in a zero-knowledge way, right? So, I don’t have to give up any privacy or raw data details, but I can prove to an auditor that we satisfy such and ...

From my personal experiments, multiple such things do stack. Actually, on a face-to-face, you can see things that are just a combination of five LoRA components, in a particular order. One very recent example is that somebody took a Mistral 7 billion, compared it to Llama 7 billion, had a ...

I’m personally quite optimistic on composable LoRA and composable fine-tuning processes. The whole point of the alignment assemblies was that each conversation either online or face-to-face for a whole day can be captured in a very long context and then distilled either constitutionally or some other way into a LoRA, ...

And I think the EU also has not backed open-source models. So, we’re now positioning open-source models as something that helps this process. Which is not a throttling mentality, this is an increasing investment on safety.

So, more like a pre-training stage measurement, and open red streaming, collaborative red streaming among the top frontier labs. So, it’s not about a cap. I don’t think White House, the cap is stopping it or pausing it. It’s just that at this level, you need to open yourself for ...

So, and Anthropic had this idea that the safety margin needed to be like six times, because the people abusing those language models, maybe they are, you know, coach members or something, they may actually be more imaginative than you and I, and have access to resources we don’t know ...

But there are common points. And by like having a lot of conversations with these clusters as part of deep canvassing, we eventually settled on two things. One is the safe.ai statement, which I signed, which says that everybody needs to take the same amount of care as toward pandemic ...

Yeah, so there’s a figure a few months ago that says there’s 30 capability researchers to everyone, safety and online and care researcher. So, there’s this huge imbalance. So, I think… Actually, we use the same technology to talk to the city at the end of March to analyze the ...

But real adversaries, especially attacking the ability, there’s no such restraints, right? If they just accidentally sabotage the computer center, totally, well, that’s maybe a win for them. So, things that can hack on its own, living off the land, it’s not sometime in the future. This is already a ...

Very much so. I was visiting Israel for Cyber Week and Pentera gave us a demo. So, Pentera is just written in a box, right? It’s a narrow AI that you just install it, it can live off the land using just the tools that this virtual machine has, connected ...

… but there’s no prompt template that can easily distinguish this. And this is actually a great example of data poisoning.

In fact, one of our MPs just yesterday made this interpolation topic to our premier, saying that no matter how he prompts DALL-E 3 or Stable Diffusion or Midjourney, if he asks for a flag of the ROC, the Republic of China, it always gives a People’s Republic of China ...

Because if it’s not so prevalent the common crawl and everything, it should be easy to get Taiwanese traditional Mandarin output. The whole difficulty of our National Academy researcher and everybody else in getting sensible traditional Mandarin output, like even when I told GPT-4 to speak in #zh-tw exclusively, I ...

The Beijing PRC ideology is a living example.

Or when an adversarial suffix is putting it into hypnosis, then you detect that hypnotizing effect and shut it down.

Ah, okay.

Yeah, the therapist fixes.

The very last slide shows this whole lifecycle of the plan. So, we collect evaluation criteria from the society, we want to automate all the eval that could be made automated, but we will keep a line open to see the society input and other dispensers that we could mention ...

But anyway, Odie has already set up such a chatbot for everybody to chat with, and soon we will pilot the test on the automatic summarization that you just saw, on public inputs, which is an interactive summarization, so to speak. So, these are the kind of uses that we ...

But we think that if it’s purely edge, like running on my laptop, or essentially in some place that has no way to connect to the Internet, then it actually inoculates the mind, it prepares the public service to the confabulations and hallucinations and things like that, as long as, ...

So, we will, at the end of the year, begin with just the language model and image classification, the easy ones, but also the ones that are actually now being used in public sector. Because we have a guideline for public sector that says if you’re in an isolated mode, ...

Evidently, people in Taiwan, for all the language models sponsored by the government, including the national academy, they insist that the traditional characters they use not just look like from Taiwan, but the vocabulary, the ideology they use may not conform to this Beijing standard. And so, the read teaming ...

So, it’s quite simple. We can do currently black tasks based on API, but we actually have the capability to do white box. So basically, look at the neurons as they’re having this conversation. But we will initially only open up this black box testing. This one is something that’s ...

And so, as you mentioned, the MITRE ATLAS or the EU guidelines usually focus on the bottom half of this chart. But actually, the people’s voice, or the people’s AI, wants it to be explainable and resilient and things like that. And it’s the grey areas, literally on the left, ...

Yeah, it’s called Alignment Assemblies. The partnership we’re in, the Collective Intelligence Project, CIP, is one of the seats at the table at the UK summit, so they’re just pushing this agenda on our behalf. And once we have such a tuned model or aligned model, and what we call ...

I had an online conversation, a Zoom meeting, between some of these people and OpenAI people. So, it’s usually three steps, right? One, with the sample citizens, setting an online agenda. Second, using the agenda to have a conversation with stakeholders. Finally, turning the result of these two steps into ...

And another platform that we use is called All Our Ideas. The idea is to rank the ideas that stand out. Like when forced to choose between two principles, what we will prefer. So, the idea of All Our Ideas is that for New York City, whether fixing streets is ...

Yeah. With the language models’ help, we start seeding some statements. People can upload or download it. People can also write their own statements for other people to vote on.

Yeah, there are many ways to do that. We primarily use Polis, which is a wiki survey. So, basically…

Try to ask the LLM very biased questions. If it all answers in a way that scores higher in terms of fairness and so on, then we say that it’s constitutionally more aligned to the original consensus. And the great thing about this is that this can be done every ...

And then, now, I think the team is synthesizing 1,000 red team things. So, for example, if one principle is one should always strive to be fair and balanced among nations, then the 100 penetration question will basically all about hate and bigotry and things like that.

Yes. And as part of the testing and verification center, as I mentioned, the conversations we held in Taiwan, including online, Taipei, and Tainan, made it possible for us to have some constitutional guidelines to the science minister’s language model. So, they’re tuning it based on that model. We also ...

Wendy here just attended Meta’s online event. They work with Stanford to basically turn such conversations into something deliberative, through Deliberative Polling®.

Yeah, instead of aggregating individual preferences, which is very suspectible to the persuasion attack you just mentioned, we instead operate on the units of statistically reasonable citizens. And that becomes a voter, not a single voter. This synthetic voter, which always goes to the source with citation and dignity and ...

Yeah, the citizen assemblies.