Large Language Models (LLM) have rapidly evolved from simple toys capable of generating stories about unicorns to powerful tools that can solve exam-level problems, advise human experts, summarize texts, and write code -- among other tasks. This remarkable advancement in LLM capabilities over the past few years has led to their frenetic deployment in production environments, manifesting a major leap in intellectual productivity. The silent threat of this breakthrough is, however, its security and privacy implications. None of these issues are currently well understood in the scientific community, causing a concerning deficit of trust in the widely deployed LLM applications. The proposed seminar is the first attempt to launch a scientific discourse among the leading researchers in AI, security, privacy, and NLP in order to systematize knowledge, assess existing systemic risks and mitigation challenges, and draw the map for future research in LLM security and privacy.

LLMs are qualitatively different from past machine learning models. They are intended to be “general purpose”: whereas an image classifier trained to classify cats from dogs obviously does not have the ability to classify cars from airplanes, a language model is expected to be useful on any question expressible through natural language. This makes it much more challenging to predict how they will be deployed, because it could be anywhere. But there are other differences, too. These models often have hundreds of billions to trillions of parameters and are kept private by companies, making it difficult to understand in what way and how they work. The loss functions optimized by LLMs are extremely complex, with local minima being a tangible threat to their generalization capability: a threat manifested in notorious “hallucination” phenomena. Last, but not least, the learning components of LLMs are not directly exposed to end users; instead, they are accessed via a dialogue interface (and its own learning mechanisms) which can also lead to “intriguing properties”.

For each of these reasons, it becomes imperative to study the security and privacy of language models in particular, beyond what is known about the security and privacy of traditional machine learning models. This Dagstuhl Seminar will address two fundamental questions:

• How secure are language models? When an adversary is present and interacts with these models, can they cause the model to behave incorrectly?
• How private are language models? Can an adversary interact with the model to violate the privacy of users?

This seminar brings together researchers from diverse backgrounds in order to study these questions. Which applications are most likely for an adversary to gain from exploiting machine learning models? Are there particular attacks that will be more prevalent than others? What defense strategies are possible—and practical? What is the cost of these defenses to utility?

In addition to these themes, the seminar will provide an open environment for the discussion of other related topics. As current research in this area is quite dynamic and explorative, we expect fruitful input from the diverse set of participants, including perspectives from academia and industry.

Creative Commons BY 4.0

Nicholas Carlini, Stephan Günnemann, Pavel Laskov, Emil C. Lupu, and Vera Rimmer