Dagstuhl Seminar 22432
Towards a Unified Model of Scholarly Argumentation
( Oct 23 – Oct 28, 2022 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/22432
Organizers
- Khalid Al-Khatib (University of Groningen, NL)
- Anita de Waard (Elsevier - Jericho, US)
- Iryna Gurevych (TU Darmstadt, DE)
- Yufang Hou (IBM Research - Dublin, IE)
Contact
- Marsha Kleinbauer (for scientific matters)
- Jutka Gasiorowski (for administrative matters)
Background
Argumentation is prevalent in scientific discourse and critical to scientific progress. Recent efforts have attempted to identify and model argumentative structures in scholarly discourse from different perspectives. Within the domain of scientific literature analysis, computational approaches to argumentation have followed the route of discourse modeling by identifying relations between spans and clauses encoding rhetorical structures (e.g., premises and conclusions), or as typed turns in community debate (e.g., supports or attacks). Another thread of research, often applied to biomedical literature, focuses on capturing functional discourse at different levels of granularity, such as objectives, methods, results or scientific claims, and their relation to reported evidence. Most work adopts a corpus perspective, either highlighting the role of sentences or phrases within the scientific discourse or aligning claims across documents, and using citations to construct claim-evidence networks that summarize the state of knowledge in a field. Within the health sciences, argumentative structures have been used to automate the production of systematic reviews by identifying key actionable knowledge elements from collections of clinical reviews, case studies, and research papers. For an overview of previous work, see e.g. [1].
Despite these varied efforts and the clear practical importance of the work, there is lack of consensus on how scientific argumentation should be formalized. For instance, it remains unclear whether formalisms popular in non-scientific domains apply to scientific discourse, and whether a single formalism can adequately support argumentation research in diverse disciplines such as biology, chemistry, materials science, medical research and computer science. This lack of consensus manifests in a dearth of shared reference corpora, which are needed to advance research into computational treatments of scientific argumentation. It has also led to the absence of an operational theory for defining argumentative components in scholarly text.
Goals
Our Dagstuhl Seminar, titled Towards a Unified Model for Scholarly Argumentation, sought to further the emergence of this missing consensus. Specifically, the seminar objectives included:
- Enabling robust advances in argument technology by collecting and working on use cases in scholarly and medical discourse;
- Starting the development of a foundational model for argumentation in science and healthcare;
- Laying the groundwork for a multidisciplinary community devoted to building and maintaining principles, tools, and models to identify key components in scholarly argumentation.
Outcomes
The seminar was attended by scientists at different levels of seniority and from a variety of research backgrounds. Some participants have made the computational modeling of argumentation or the scholarly literature the central focus of their careers. Others were drawn to the seminar through their work on applications in adjacent problem areas. Ultimately, all emerged with a sense that important bonds of shared interest had formed, fostered by several seminar outcomes.
Knowledge Baselining
A shared understanding of the problem space was obtained, through a series of keynotes and panel discussions on theory, models, tools, and available corpora. These are described in greater detail in this report, in Section 3. In particular, two introductory talks summarized the state of the art in argument modeling (3.1) and computational argument mining (3.2).
Five further plenary talks described different use cases where argument identification can support NLP tasks:
- using scientific discourse to understand and measure the impact of scholarly contributions (3.3);
- using argument modeling to generate discourse (3.4);
- generating scholarly documents using argument structures (3.5);
- interpreting a fortiori arguments (3.6);
- synthesizing evidence from text to support public policy (3.7).
A series of eleven flash talks covered a host of other efforts, presenting corpora, tools, and relevant applications, such as document understanding, extracting high-level claims, and identifying fallacious and persuasive elements in scholarly texts (Section 4).
Problem Elucidation
At the beginning of the workshop, the group identified several important focus areas that then became the subject of breakout group deliberation over the course of the week. All materials, including the full program, slides, summaries of the breakout sessions and code and corpora submitted can be found on the workshop Google Drive at https://bit.ly/TUMSA22.
- Foundations (Section 5.1). A subgroup of participants discussed a shared argumentation model, based on the various proposals presented during the plenary sessions. The group debated and wrote a first-order consensus of these varying views, which can be used for further development of a foundational model of scholarly argumentation.
- Domains (Section 5.2). This working group pursued a comparison of argumentation in different scholarly domains. A methodology was delineated for how to annotate argumentation across domains while reducing the need for domain experts.
- Argument Quality (Section 5.3). This working group explored how argumentation quality can be evaluated, and defined a series of questions to assess this. Additionally, the group members contributed an open-source tool to perform the evaluation of argumentation quality, which can be further developed to support this task.
- Community Dialogue (Section 5.4). This working group looked at how argument structure can support an important editorial task, namely to decide on an accept or reject decision for a submitted manuscript, based on a number of peer reviews. The group developed a corpus of dialogues that simulate how a meta-reviewer asks questions about a document that has received a number of reviews, which can be used in future work in this domain.
Community Formation
Building on the connections developed during the seminar, a series of collaborations have been fostered, and thoughts on how to proceed with this work through a multidisciplinary lens have been put forth. Multiple new collaborations have been formed as an outcome of this week, in some cases centered on new tools and research corpora first conceived in the workshop.
Next Steps
This Dagstuhl Seminar brought together a multi-disciplinary, international, and diverse community of researchers from academia and industry to discuss scholarly argumentation. Much argumentation occurred, during and after presentations, in breakout groups, during the social events spread out through the week, and long into the night. Necessarily, this is only the beginning of a conversation that will unfold over the coming years, one that will ultimately produce a shared model of scholarly argumentation and a set of concrete research tasks and important new use cases.
We hope that this seminar was the first in a series of events devoted to this topic, that this inaugural event proves pivotal in the formation of a cohesive research community addressing a problem with large practical ramifications. This report can hopefully contribute to accelerate work in this area, by offering a summary of current efforts, and a number of interesting problems to work on.
References
- Khalid Al Khatib, Tirthankar Ghosal, Yufang Hou, Anita de Waard, and Dayne Freitag. 2021. Argument Mining for Scholarly Document Processing: Taking Stock and Looking Ahead. In Proceedings of the Second Workshop on Scholarly Document Processing, pages 56–65, Online. Association for Computational Linguistics.
Creative Commons BY 4.0
Khalid Al-Khatib, Anita de Waard, Iryna Gurevych, and Yufang Hou
Argumentation is prevalent in scientific discourse and critical to scientific progress. Recent efforts have attempted to identify and model argumentative structures in the scientific literature, but from a diversity of perspectives:
- The scientific literature is a canonical domain in work on computational accounts of argumentation which attempt to model relations between spans and clauses encoding rhetorical structures (e.g., premises and conclusions) or community debate (e.g., supports or attacks).
- Another thread of research, sometimes applied to bioinformatics, focuses on scientific claims and their relation to reported evidence. Much of this work adopts a corpus perspective, aligning claims across documents, using citations to construct claim-evidence networks that summarize the state of knowledge in a field.
- Mainly within the health sciences, argumentative structures are being exploited to automate the production of systematic reviews, by identifying key actionable knowledge elements from collections of clinical reviews, case studies, and the scientific literature.
Despite this abundance of interest and the clear practical importance of the work, we lack consensus on how scientific argumentation should be formalized. It remains unclear whether formalisms popular in non-scientific domains can be adapted for scientific discourse – or even whether a single formalism can adequately support argumentation research in literatures as diverse as biology, chemistry, materials science, and medical science. This manifests in a dearth of shared reference corpora needed to advance research into computational treatments of scientific argumentation, and no consensus on a model for defining argumentative components in scholarly text.
The purpose of this Dagstuhl Seminar is threefold:
- Lay the groundwork for a nascent, multidisciplinary community devoted to building and maintaining principles, tools, and models to identify key components in scholarly argumentation;
- Develop a foundational model for argumentation in science and healthcare, in order to
- Enable robust advances in argument technology for scholarly and medical discourse.
Prior to the seminar, participants will be invited to contribute to the seminar preparation, e.g., by submitting reference material, submitting any corpora and formal descriptions suitable for use in an annotation exercise, or by attending tutorial sessions.
The seminar itself is scheduled to take place over five days, and has the following objectives:
- Knowledge baselining. We will foster a shared understanding of the problem space through a series of keynotes and panel discussions on theory, models, tools, and available corpora.
- Model discussion. Participants will discuss the models introduced during baselining, examining their adequacy, generality, and possibilities for unification under a generalized model.
- Pilot annotation. Breakout groups will apply these models to sample texts, investigating ease of annotation, completeness, and approaches to assessing reliability.
- Application discussion. Building on insights acquired during the seminar, we will discuss the amenability of available models to automated argument mining in various scientific domains and the appropriate success metrics.
- Synthesis. We will draft main conclusions and recommendations to the broader community, identify lingering open questions, and discuss future workshops and shared tasks.
Anita de Waard, Iryna Gurevych, Yufang Hou, and Khalid Al-Khatib
- Khalid Al-Khatib (University of Groningen, NL) [dblp]
- Milad Alshomary (Leibniz Universität Hannover, DE)
- Wolf-Tilo Balke (TU Braunschweig, DE) [dblp]
- Tilman Beck (TU Darmstadt, DE)
- Elena Cabrio (Université Côte d’Azur - Sophia Antipolis, FR) [dblp]
- Fengyu Cai (TU Darmstadt, DE)
- Davide Ceolin (CWI - Amsterdam, NL)
- Anita de Waard (Elsevier - Jericho, US) [dblp]
- Nils Dycke (TU Darmstadt, DE)
- Dayne Freitag (SRI - Menlo Park, US)
- Daniel Garijo (Polytechnic University of Madrid, ES) [dblp]
- Iryna Gurevych (TU Darmstadt, DE) [dblp]
- Graeme Hirst (University of Toronto, CA) [dblp]
- Yufang Hou (IBM Research - Dublin, IE) [dblp]
- Eduard H. Hovy (Carnegie Mellon University, Pittsburgh, US & University of Melbourne, AU) [dblp]
- Anne Lauscher (Universität Hamburg, DE)
- Maria Liakata (Queen Mary University of London, GB)
- Tobias Mayer (TU Darmstadt, DE)
- Robert Mercer (University of Western Ontario - London, CA) [dblp]
- Smaranda Muresan (Columbia University - New York, US) [dblp]
- Preslav Nakov (MBZUAI - Abu Dhabi, AE) [dblp]
- Sukannya Purkayastha (TU Darmstadt, DE)
- Chris Reed (University of Dundee, GB) [dblp]
- Domenic Rosati (scite - Halifax, CA)
- Florian Ruosch (Universität Zürich, CH)
- Harrisen Scells (Universität Leipzig, DE)
- Ferdinand Schlatt (Universität Halle-Wittenberg, DE)
- Benno Stein (Bauhaus-Universität Weimar, DE) [dblp]
- Simone Teufel (University of Cambridge, GB) [dblp]
- Serena Villata (Université Côte d’Azur - Sophia Antipolis, FR) [dblp]
- Andreas Vlachos (University of Cambridge, GB) [dblp]
- Henning Wachsmuth (Universität Paderborn, DE) [dblp]
- Ryan Wang (University of Illinois at Urbana Champaign, US)
Classification
- Artificial Intelligence
- Computation and Language
- Machine Learning
Keywords
- Argument mining
- argument modeling
- scholarly discourse
