Dagstuhl-Seminar 19461
Conversational Search
( 10. Nov – 15. Nov, 2019 )
(zum Vergrößern in der Bildmitte klicken)
Permalink
Bitte benutzen Sie folgende Kurz-Url zum Verlinken dieser Seite:
https://www.dagstuhl.de/19461
Organisatoren
- Avishek Anand (Leibniz Universität Hannover, DE)
- Lawrence Cavedon (RMIT University - Melbourne, AU)
- Hideo Joho (University of Tsukuba - Ibaraki, JP)
- Mark Sanderson (RMIT University - Melbourne, AU)
- Benno Stein (Bauhaus-Universität Weimar, DE)
Kontakt
- Michael Gerke (für wissenschaftliche Fragen)
- Susanne Bach-Bernhard (für administrative Fragen)
Impacts
- A Large-Scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search : article - Vakulenko, Svitlana; Kanoulas, Evangelos; Rijke, Maarten de - Cornell University : arXiv.org, 2021. - 32 pp..
- Common Conversational Community Prototype : Scholarly Conversational Assistant - Balog, Krisztian; Flekova, Lucie; Radlinski, Filip; Vakulenko, Svitlana; Zamani, Hamed; Sanderson, Mark; Potthast, Martin; Jones, Rosie; Hagen, Matthias - Cornell University : arXiv.org, 2020. - 6 pp..
- Dagstuhl Seminar 19461 on Conversational Search : Seminar Goals and Working Group Outcomes : Article - Anand, Avishek; Cavedon, Lawrence; Hagen, Matthias; Joho, Hideo; Sanderson, Mark; Stein, Benno M. - New York : ACM, 2020. - 11 pp. - (Sigir forum ; 54. 2020, 1).
Programm
The Conversational Search Paradigm promises to satisfy information needs using human-like dialogs, be it in spoken or in written form. This kind of “information-providing dialogs” will increasingly happen enpassant and spontaneously, probably triggered by smart objects with which we are surrounded such as intelligent assistants (Alexa, Siri, etc.), domestic appliances, environmental control devices, toys, or autonomous robots and vehicles. The outlined development marks a paradigm shift for information technology, and the key question(s) is (are):
What does Conversational Search mean and how to make the most of it—given the possibilities and the restrictions that come along with this paradigm?
Currently, our understanding is still too limited to exploit the Conversational Search Paradigm for effectively satisfying the existing diversity of information needs. Hence, with this first Dagstuhl Seminar on Conversational Search we intend to bring together leading researchers from relevant communities to understand and to analyze this promising retrieval paradigm and its future from different angles.
Among others, we expect to discuss issues related to interactivity, result presentation, clarification, user models, and evaluation, but also search behavior that can lead into a human-machine debate or an argumentation related to the information need in question.
Moreover, we expect to define, shape, and formalize a set of corresponding problems to be addressed, as well as to highlight associated challenges that are expected to come in the form of multiple modalities and multiple users. Correspondingly, we intend to define a roadmap for establishing a new interdisciplinary research community around Conversational Search, for which the seminar will serve as a prominent scientific event, with hopefully many future events to come.
Creative Commons BY 3.0 DE
Avishek Anand, Hideo Joho, Mark Sanderson, and Benno Stein
Background and Motivation
The Conversational Search Paradigm promises to satisfy information needs using human-like dialogs, be it in spoken or in written form. This kind of "information-providing dialogs" will increasingly happen enpassant and spontaneously, probably triggered by smart objects with which we are surrounded such as intelligent assistants such as Amazon Alexa, Apple Siri, Google Assistant, and Microsoft Cortana, domestic appliances, environmental control devices, toys, or autonomous robots and vehicles. The outlined development marks a paradigm shift for information technology, and the key question(s) is (are):
What does Conversational Search mean and how to make the most of it--given the possibilities and the restrictions that come along with this paradigm?
Currently, our understanding is still too limited to exploit the Conversational Search Paradigm for effectively satisfying the existing diversity of information needs. Hence, with this first Dagstuhl Seminar on Conversational Search we intend to bring together leading researchers from relevant communities to understand and to analyze this promising retrieval paradigm and its future from different angles.
Among others, we expect to discuss issues related to interactivity, result presentation, clarification, user models, and evaluation, but also search behavior that can lead into a human-machine debate or an argumentation related to the information need in question.
Moreover, we expect to define, shape, and formalize a set of corresponding problems to be addressed, as well as to highlight associated challenges that are expected to come in the form of multiple modalities and multiple users. Correspondingly, we intend to define a roadmap for establishing a new interdisciplinary research community around Conversational Search, for which the seminar will serve as a prominent scientific event, with hopefully many future events to come.
Seminar Program
A 5-day program of the seminar consisted of six introductory and background sessions, three visionary talk sessions, one industry talk session, and nine breakout discussion and reporting sessions. The seminar also had three social events during the program. The detail program of the seminar is available online.
Pre-Seminar Activities
Prior to the seminar, participants were asked to provide inputs to the following questions and request:
- What are your ideas of the "ultimate" conversational search system?
- Please list, from the perspective of your research field, important open questions or challenges in conversational search.
- What are the three papers a PhD student in conversational search should read and why?
From the survey, the following topics were initially emerged as interests of participants. Many of these topics were discussed at length in the seminar.
- Understanding nature of information seeking in the context of conversational agents
- Modelling problems in conversational search
- Clarification and explanation
- Evaluation in conversational search systems
- Ethics and privacy in conversational systems
- Extending the problem space beyond the search interface and Q/A
Another outcome of the above pre-seminar questions was a compilation of recommended reading list to gain a solid understanding of topics and technologies that were related to the research on Conversational Search. The reading list is provided in Section 5 of this report.
Invited Talks
One of the main goals and challenges of this seminar was to bring a broad range of researchers together to discuss Conversational Search, which required to establish common terminologies among participants. Therefore, we had a series of 18 invited talk throughout the seminar program to facilitate the understanding and discussion of conversational search and its potential enabling technologies. The main part of this report includes the abstract of all talks.
Working Groups
In the afternoon of Day 2, initial working groups were formed based on the inputs to the pre-seminar questionnaires, introductory and background talks, and discussions among participants. On Day 3, the grouping was revisited and updated, and, eventually, the following seven groups were formed to focus on topics such as the definition, evaluation, modelling, explanation, scenarios, applications, and prototype of Conversational Search.
- Defining Conversational Search
- Evaluating Conversational Search
- Modeling in Conversational Search
- Argumentation and Explanation
- Scenarios that Invite Conversational Search
- Conversation Search for Learning Technologies
- Common Conversational Community Prototype: Scholarly Conversational Assistant
We have summarized the working groups' outcomes in the following. Please refer to the main part of this report for the full description of the findings.
Defining Conversational Search
This group aimed to bring structure and common terminology to the different aspects of conversational search systems that characterise the field. After reviewing existing concepts such as Conversational Answer Retrieval and Conversational Information Seeking, the group offers a typology of Conversational Search systems via functional extensions of information retrieval systems, chatbots, and dialogue systems. The group further elaborates the attributes of Conversational Search by discussing its dimensions and desirable additional properties. Their report suggests types of systems that should not be confused as conversational search systems.
Evaluating Conversational Search
This group addressed how to determine the quality of conversational search for evaluation. They first describe the complexity of conversation between search systems and users, followed by a discussion of the motivation and broader tasks as the context of conversational search that can inform the design of conversational search evaluation. The group also surveys 12 recent tasks and datasets that can be exploited for evaluation of conversational search. Their report presents several dimensions in the evaluation such as User, Retrieval, and Dialog, and suggests that the dimensions might have an overlap with those of Interactive Information Retrieval.
Modeling Conversational Search
This group addressed what should be modeled from the real world to achieve a successful conversational search and how. They explain why a range of concepts and variables such as capabilities and resources of systems, beliefs and goals of users, history and current status of process, and search topics and tasks should be considered to advance understanding between systems and users in the context of Conversational Search. The group points out that the options the current search engines present to users can be too broad in conversational interaction. They suggest that a deeper modeling of users' beliefs and wants, development of reflective mechanisms, and finding a good balance between macroscopic and microscopic modeling are promising directions for future research.
Argumentation and Explanation
Motivated by inevitable influences made to users due to the course of actions and choices of search engines, this group explored how the research on argumentation and explanation can mitigate some of potential biases generated during conversational search processes, and facilitate users' decision-making by acknowledging different viewpoints of a topic. The group suggests a research scheme that consists of three layers: a conversational layer, a demographics layer, and a topic layer. Also, their report explains that argumentation and explanation should be carefully considered when search systems (1) select, (2) arrange, and (3) phrase the information presented to the users. Creating an annotated corpus with these elements is the next step in this direction.
Scenarios for Conversational Search
This group aimed to identify scenarios that invite conversational search, given that natural language conversation might not always be the best way to search in some context. Their report summarises that modality and task of search are the two cases where conversational search might make sense. Modality can be determined by a situation such as driving or cooking, or devices at hand such as a smartwatch or AR/VR systems. As for the task, the group explains that the usefulness of conversational search increases as the level of exploration and complexity increases in tasks. On the other hand, simple information needs, highly ambiguous situations, or very social situations might not be the bast case for conversational search. Proposed scenarios include a mechanic fixing a machine, two people searching for a place for dinner, learning about a recent medical diagnosis, and following up on a news article to learn more.
Conversation Search for Learning Technologies
This group discussed the implication of conversational search from learning perspectives. The report highlights the importance of search technologies in lifelong learning and education, and the challenges due to complexity of learning processes. The group points out that multimodal interaction is particularly useful for educational and learning goals since it can support students with diverse background. Based on these discussions, the report suggests several research directions including extension of modalities to speech, writing, touch, gaze, and gesturing, integration of multimodal inputs/outputs with existing IR techniques, and application of multimodal signals to user modelling.
Common Conversational Community Prototype: Scholarly Conversational Assistant
This group proposed to develop and operate a prototype conversational search system for scholarly activities as academic resources that support research on conversational search. Example activities include finding articles for a new area of interest, planning sessions to attend in a conference, or determining conference PC members. The proposed prototype is expected to serve as a useful search tool, a means to create datasets, and a platform for community-based evaluation campaigns. The group outlined also a road map of the development of a Scholarly Conversational Assistant. The report includes a set of software platforms, scientific IR tools, open source conversational agents, and data collections that can be exploited in conversational search work.
Conclusions
Leading researchers from diverse domains in academia and industries investigated the essence, attributes, architecture, applications, challenges, and opportunities of Conversational Search in the seminar. One clear signal from the seminar is that research opportunities to advance Conversational Search are available to many areas and collaboration in an interdisciplinary community is essential to achieve the goal. This report should serve as one of the main sources to facilitate such diverse research programs on Conversational Search.
Avishek Anand, Lawrence Cavedon, Hideo Joho, Mark Sanderson, and Benno Stein
- Khalid Al-Khatib (Bauhaus-Universität Weimar, DE) [dblp]
- Avishek Anand (Leibniz Universität Hannover, DE) [dblp]
- Elisabeth André (Universität Augsburg, DE) [dblp]
- Jaime Arguello (University of North Carolina - Chapel Hill, US) [dblp]
- Leif Azzopardi (University of Strathclyde - Glasgow, GB) [dblp]
- Krisztian Balog (University of Stavanger, NO) [dblp]
- Nicholas J. Belkin (Rutgers University - New Brunswick, US) [dblp]
- Robert Capra (University of North Carolina - Chapel Hill, US) [dblp]
- Lawrence Cavedon (RMIT University - Melbourne, AU) [dblp]
- Leigh Clark (Swansea University, GB) [dblp]
- Phil Cohen (Monash University - Clayton, AU)
- Ido Dagan (Bar-Ilan University - Ramat Gan, IL) [dblp]
- Arjen P. de Vries (Radboud University Nijmegen, NL) [dblp]
- Ondrej Dusek (Charles University - Prague, CZ) [dblp]
- Jens Edlund (KTH Royal Institute of Technology - Stockholm, SE) [dblp]
- Lucie Flekova (Amazon R&D - Aachen, DE) [dblp]
- Bernd Fröhlich (Bauhaus-Universität Weimar, DE) [dblp]
- Norbert Fuhr (Universität Duisburg-Essen, DE) [dblp]
- Ujwal Gadiraju (Leibniz Universität Hannover, DE) [dblp]
- Matthias Hagen (Martin-Luther-Universität Halle-Wittenberg, DE) [dblp]
- Claudia Hauff (TU Delft, NL) [dblp]
- Gerhard Heyer (Universität Leipzig, DE) [dblp]
- Hideo Joho (University of Tsukuba - Ibaraki, JP) [dblp]
- Rosie Jones (Spotify - Boston, US) [dblp]
- Ronald M. Kaplan (Stanford University, US) [dblp]
- Mounia Lalmas-Rölleke (Spotify - London, GB) [dblp]
- Jurek Leonhardt (Leibniz Universität Hannover, DE) [dblp]
- David Maxwell (University of Glasgow, GB) [dblp]
- Sharon Oviatt (Monash University - Clayton, AU) [dblp]
- Martin Potthast (Universität Leipzig, DE) [dblp]
- Filip Radlinski (Google UK - London, GB) [dblp]
- Rishiraj Saha Roy (MPI für Informatik - Saarbrücken, DE) [dblp]
- Mark Sanderson (RMIT University - Melbourne, AU) [dblp]
- Ruihua Song (Microsoft XiaoIce- Beijing, CN) [dblp]
- Laure Soulier (UPMC - Paris, FR) [dblp]
- Benno Stein (Bauhaus-Universität Weimar, DE) [dblp]
- Markus Strohmaier (RWTH Aachen, DE) [dblp]
- Idan Szpektor (Google Israel - Tel-Aviv, IL) [dblp]
- Jaime Teevan (Microsoft Corporation - Redmond, US) [dblp]
- Johanne Trippas (RMIT University - Melbourne, AU) [dblp]
- Svitlana Vakulenko (Wirtschaftsuniversität Wien, AT) [dblp]
- Henning Wachsmuth (Universität Paderborn, DE) [dblp]
- Emine Yilmaz (University College London, GB) [dblp]
- Hamed Zamani (Microsoft Corporation, US) [dblp]
Klassifikation
- artificial intelligence / robotics
- data bases / information retrieval
- society / human-computer interaction
Schlagworte
- Information Retrieval
- Discourse and Dialogue
- Interactive Systems
- Human-Machine Interaction
- User Simulation
